[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:14:57 dist-test-slave-gjrz krb5kdc[12772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:14:57 dist-test-slave-gjrz krb5kdc[12772](info): set up 2 sockets
Aug 10 07:14:57 dist-test-slave-gjrz krb5kdc[12772](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 07:14:59.006093 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:14:59.122598 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44635
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:59.236600 12791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:59.236831 12791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:59.236882 12791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:59.240784 12791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:14:59.240835 12791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:59.240861 12791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:59.240881 12791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:14:59.240907 12791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:14:59.245954 12791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:14:59.246974 12791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:59.248015 12791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:59.256172 12801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.256220 12798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.256186 12804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.256491 12791 server_base.cc:1034] running on GCE node
I20260810 07:14:59.257018 12791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:59.258317 12791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:59.259474 12791 hybrid_clock.cc:648] HybridClock initialized: now 1786346099259451 us; error 68 us; skew 500 ppm
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:59.262228 12791 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:14:59.263051 12791 webserver.cc:460] Webserver started at http://127.12.117.190:33255/ using document root <none> and password file <none>
I20260810 07:14:59.263650 12791 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:59.263715 12791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:59.263979 12791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:59.265875 12791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "f75f60ebe9e9996d3036d20d8894d611"
server_key_iv: "f93f254b119392455c0c70a7a4ecee82"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.266379 12791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "f75f60ebe9e9996d3036d20d8894d611"
server_key_iv: "f93f254b119392455c0c70a7a4ecee82"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.269795 12791 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:14:59.272063 12818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.273265 12791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:14:59.273396 12791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "f75f60ebe9e9996d3036d20d8894d611"
server_key_iv: "f93f254b119392455c0c70a7a4ecee82"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.273490 12791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:59.291790 12791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:59.294682 12791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:59.294858 12791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:59.301831 12791 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:44635
I20260810 07:14:59.301855 12896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:44635 every 8 connection(s)
I20260810 07:14:59.302755 12791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:14:59.305827 12898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:14:59.308670 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12791
I20260810 07:14:59.308830 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:14:59.310449 12758 external_mini_cluster.cc:1428] Setting key dd754ac1c3c3b3471a1cf827a2befc3b
I20260810 07:14:59.312503 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: Bootstrap starting.
I20260810 07:14:59.314949 12898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: Neither blocks nor log segments found. Creating new log.
I20260810 07:14:59.315730 12898 log.cc:824] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:14:59.317539 12898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: No bootstrap required, opened a new log
I20260810 07:14:59.322053 12898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } }
I20260810 07:14:59.322307 12898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:14:59.322364 12898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fecb6b7b8141e68e6cc66b81bc9fdb, State: Initialized, Role: FOLLOWER
I20260810 07:14:59.322814 12898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } }
I20260810 07:14:59.322962 12898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:14:59.323035 12898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:14:59.323137 12898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:14:59.323972 12898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } }
I20260810 07:14:59.324291 12898 leader_election.cc:302] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89fecb6b7b8141e68e6cc66b81bc9fdb; no voters:
I20260810 07:14:59.324607 12898 leader_election.cc:288] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:14:59.324702 12906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:14:59.324935 12906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [term 1 LEADER]: Becoming Leader. State: Replica: 89fecb6b7b8141e68e6cc66b81bc9fdb, State: Running, Role: LEADER
I20260810 07:14:59.325265 12906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } }
I20260810 07:14:59.325855 12898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:14:59.327023 12908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89fecb6b7b8141e68e6cc66b81bc9fdb. Latest consensus state: current_term: 1 leader_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } } }
I20260810 07:14:59.327054 12907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89fecb6b7b8141e68e6cc66b81bc9fdb" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44635 } } }
I20260810 07:14:59.327153 12907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [sys.catalog]: This master's current role is: LEADER
I20260810 07:14:59.327152 12908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb [sys.catalog]: This master's current role is: LEADER
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
W20260810 07:14:59.331506 12924 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:14:59.331581 12924 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:14:59.331642 12915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:14:59.332423 12915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:14:59.336035 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.314114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.12.117.190:44635)
0810 07:14:59.314571 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:59.314596 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:59.315637 (+ 1041us) server_negotiation.cc:407] Connection header received
0810 07:14:59.318683 (+ 3046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:59.318705 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:59.319090 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:59.319468 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:59.321420 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.322959 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.324266 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.324934 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.326377 (+ 1443us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.329935 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:59.329941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:59.329958 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:59.329995 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:59.331968 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.332600 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.332606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.332611 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.332750 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.333183 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.333187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.333189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.333771 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:59.333982 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:59.334586 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 07:14:59.334845 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":138,"threads_started":1}
I20260810 07:14:59.338701 12915 catalog_manager.cc:1347] Generated new cluster ID: f842d0f44b9342cfaf9408446e9f4e08
I20260810 07:14:59.338754 12915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:14:59.361467 12915 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:14:59.362375 12915 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:14:59.366797 12915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89fecb6b7b8141e68e6cc66b81bc9fdb: Generated new TSK 0
I20260810 07:14:59.367527 12915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:14:59.404309 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:59.555400 12933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:59.555635 12933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:59.555696 12933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:59.559573 12933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:59.559626 12933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:59.559703 12933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:14:59.564855 12933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:14:59.565778 12933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:59.566708 12933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:59.576453 12941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.576453 12942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.576450 12945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.576558 12933 server_base.cc:1034] running on GCE node
I20260810 07:14:59.577064 12933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:59.577713 12933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:59.578881 12933 hybrid_clock.cc:648] HybridClock initialized: now 1786346099578861 us; error 30 us; skew 500 ppm
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:59.581629 12933 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:14:59.582476 12933 webserver.cc:460] Webserver started at http://127.12.117.129:35795/ using document root <none> and password file <none>
I20260810 07:14:59.583001 12933 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:59.583070 12933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:59.583307 12933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:59.585245 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "2c39f5bd3eb44a2898865ae84af06748"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "28fb3d8d497706b0047d03cbf26e1c52"
server_key_iv: "7581077bc78e79f785f842df6eace0d0"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.585778 12933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "2c39f5bd3eb44a2898865ae84af06748"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "28fb3d8d497706b0047d03cbf26e1c52"
server_key_iv: "7581077bc78e79f785f842df6eace0d0"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.589371 12933 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:14:59.591730 12960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.592875 12933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:14:59.592983 12933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "2c39f5bd3eb44a2898865ae84af06748"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "28fb3d8d497706b0047d03cbf26e1c52"
server_key_iv: "7581077bc78e79f785f842df6eace0d0"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.593080 12933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:59.611795 12933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:59.614326 12933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:59.614497 12933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:59.615021 12933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:59.615811 12933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:59.615859 12933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.615909 12933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:59.615939 12933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.624586 12933 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:37565
I20260810 07:14:59.624614 13153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:37565 every 8 connection(s)
I20260810 07:14:59.625358 12933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:14:59.630977 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12933
I20260810 07:14:59.631067 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:14:59.631321 12758 external_mini_cluster.cc:1428] Setting key 02d117a7635d2c9a2e5729e1d8443678
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:14:59.639225 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.627097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:37975 (local address 127.12.117.190:44635)
0810 07:14:59.627240 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:59.627245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:59.628134 (+ 889us) server_negotiation.cc:407] Connection header received
0810 07:14:59.629572 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:59.629575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:59.629623 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:59.629698 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:59.630783 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.631433 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.632874 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.633132 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.633210 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.635857 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:59.635861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:59.635864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:59.635890 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:59.637283 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.637764 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.637767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.637768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.637847 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.638166 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.638170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.638172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.638346 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:59.638431 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:59.638914 (+ 483us) server_negotiation.cc:299] Negotiation successful
0810 07:14:59.639074 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:14:59.639945 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.627473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44635 (local address 127.12.117.129:37975)
0810 07:14:59.627956 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:59.628016 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:59.629353 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:59.629850 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:59.629857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:59.630209 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:59.630646 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:59.630659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.631559 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.631562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:59.632759 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:59.632769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.633285 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.633291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:59.633468 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.633985 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:59.634012 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:59.635686 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:59.637403 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:59.637407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:59.637419 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:59.637661 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:59.637936 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:59.637939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:59.637941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:59.638071 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:59.638453 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:59.638459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:59.638687 (+ 228us) client_negotiation.cc:769] Sending connection context
0810 07:14:59.638847 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:14:59.639139 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":136,"threads_started":1}
I20260810 07:14:59.641191 13155 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44635
I20260810 07:14:59.641441 13155 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:14:59.641999 13155 heartbeater.cc:511] Master 127.12.117.190:44635 requested a full tablet report, sending...
I20260810 07:14:59.643517 12847 ts_manager.cc:194] Registered new tserver with Master: 2c39f5bd3eb44a2898865ae84af06748 (127.12.117.129:37565)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:14:59.644780 12847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:37975
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:14:59.657761 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:59.765807 13161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:59.766033 13161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:59.766080 13161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:59.769873 13161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:59.769917 13161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:59.769970 13161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:14:59.774873 13161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:14:59.775784 13161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:59.776688 13161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:59.785449 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.785506 13161 server_base.cc:1034] running on GCE node
W20260810 07:14:59.785670 13174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.785449 13171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.786068 13161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:59.786638 13161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:59.787796 13161 hybrid_clock.cc:648] HybridClock initialized: now 1786346099787778 us; error 33 us; skew 500 ppm
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:59.790330 13161 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:14:59.791143 13161 webserver.cc:460] Webserver started at http://127.12.117.130:35787/ using document root <none> and password file <none>
I20260810 07:14:59.791683 13161 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:59.791752 13161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:59.791932 13161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:59.793816 13161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "bdace17ed51b40f9a279b6b708a29fdf"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "59128700ea1069cb008ac42eafee8ea3"
server_key_iv: "e547a7030c46a80e33ccdaed335706c8"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.794349 13161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "bdace17ed51b40f9a279b6b708a29fdf"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "59128700ea1069cb008ac42eafee8ea3"
server_key_iv: "e547a7030c46a80e33ccdaed335706c8"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.797850 13161 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:14:59.800099 13185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.801213 13161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:14:59.801318 13161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "bdace17ed51b40f9a279b6b708a29fdf"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "59128700ea1069cb008ac42eafee8ea3"
server_key_iv: "e547a7030c46a80e33ccdaed335706c8"
server_key_version: "encryptionkey@0"
I20260810 07:14:59.801414 13161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:14:59.817572 13161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:14:59.820441 13161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:14:59.820611 13161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:14:59.821144 13161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:14:59.821934 13161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:14:59.821983 13161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.822033 13161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:14:59.822068 13161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:14:59.831357 13161 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:43063
I20260810 07:14:59.831393 13393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:43063 every 8 connection(s)
I20260810 07:14:59.832183 13161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:14:59.833469 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13161
I20260810 07:14:59.833561 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:14:59.833823 12758 external_mini_cluster.cc:1428] Setting key 7338ad2ac03a43e12aa0ee0485c4a489
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:14:59.851833 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.833837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:48229 (local address 127.12.117.190:44635)
0810 07:14:59.834311 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 07:14:59.834316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:14:59.834819 (+ 503us) server_negotiation.cc:407] Connection header received
0810 07:14:59.836357 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:14:59.836360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:14:59.836442 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:14:59.836526 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:14:59.838042 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.838847 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.842179 (+ 3332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.842526 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.842619 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.847436 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:14:59.847441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:14:59.847445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:14:59.847479 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:14:59.849585 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.850274 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.850278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.850280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.850375 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:14:59.850717 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:14:59.850719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:14:59.850721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:14:59.850906 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:14:59.850980 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:14:59.851603 (+ 623us) server_negotiation.cc:299] Negotiation successful
0810 07:14:59.851711 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379}
I20260810 07:14:59.852735 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:14:59.834203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44635 (local address 127.12.117.130:48229)
0810 07:14:59.834659 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0810 07:14:59.834711 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:14:59.836123 (+ 1412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:14:59.836799 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:14:59.836808 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:14:59.837248 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:14:59.837890 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:59.837910 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.840285 (+ 2375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.840288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:59.842034 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:14:59.842048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:14:59.842773 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:14:59.842777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:14:59.842950 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:14:59.845279 (+ 2329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:14:59.845298 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:14:59.847164 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:14:59.849708 (+ 2544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:59.849714 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:59.849730 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:59.850070 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:59.850457 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:14:59.850459 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:14:59.850460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:14:59.850604 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:14:59.851013 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:14:59.851020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:14:59.851349 (+ 329us) client_negotiation.cc:769] Sending connection context
0810 07:14:59.851553 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 07:14:59.851882 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":130,"threads_started":1}
I20260810 07:14:59.853891 13394 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44635
I20260810 07:14:59.854117 13394 heartbeater.cc:464] Registering TS with master...
I20260810 07:14:59.854585 13394 heartbeater.cc:511] Master 127.12.117.190:44635 requested a full tablet report, sending...
I20260810 07:14:59.855638 12847 ts_manager.cc:194] Registered new tserver with Master: bdace17ed51b40f9a279b6b708a29fdf (127.12.117.130:43063)
I20260810 07:14:59.856196 12847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:48229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:14:59.864238 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:14:59.970981 13402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:14:59.971202 13402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:14:59.971256 13402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:14:59.975183 13402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:14:59.975234 13402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:14:59.975334 13402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:14:59.980566 13402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:14:59.981429 13402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:14:59.982360 13402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:14:59.991078 13410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:14:59.991102 13411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.991199 13402 server_base.cc:1034] running on GCE node
W20260810 07:14:59.991111 13413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:14:59.991832 13402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:14:59.992352 13402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:14:59.993530 13402 hybrid_clock.cc:648] HybridClock initialized: now 1786346099993510 us; error 31 us; skew 500 ppm
Aug 10 07:14:59 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:14:59.996172 13402 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:14:59.996960 13402 webserver.cc:460] Webserver started at http://127.12.117.131:40669/ using document root <none> and password file <none>
I20260810 07:14:59.997489 13402 fs_manager.cc:359] Metadata directory not provided
I20260810 07:14:59.997551 13402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:14:59.997710 13402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:14:59.999583 13402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "2db5865a371f426b891e64d8078f969f"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "0eea1e72a91153e916512635ed469cac"
server_key_iv: "5a927e0153577325fd786027816ead61"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.000072 13402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "2db5865a371f426b891e64d8078f969f"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "0eea1e72a91153e916512635ed469cac"
server_key_iv: "5a927e0153577325fd786027816ead61"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.003480 13402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:15:00.005746 13425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.006860 13402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:00.006992 13402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "2db5865a371f426b891e64d8078f969f"
format_stamp: "Formatted at 2026-08-10 07:14:59 on dist-test-slave-gjrz"
server_key: "0eea1e72a91153e916512635ed469cac"
server_key_iv: "5a927e0153577325fd786027816ead61"
server_key_version: "encryptionkey@0"
I20260810 07:15:00.007076 13402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:00.021653 13402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:00.024060 13402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:00.024192 13402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:00.024657 13402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:00.025348 13402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:00.025394 13402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.025429 13402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:00.025447 13402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:00.033951 13402 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:43197
I20260810 07:15:00.033981 13617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:43197 every 8 connection(s)
I20260810 07:15:00.034741 13402 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:00.039216 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13402
I20260810 07:15:00.039356 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:00.039638 12758 external_mini_cluster.cc:1428] Setting key 24c03458833b79c33c7b0c1fc76cb686
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346099, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:00.047715 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.036469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:40157 (local address 127.12.117.190:44635)
0810 07:15:00.036588 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:00.036591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:00.037344 (+ 753us) server_negotiation.cc:407] Connection header received
0810 07:15:00.038557 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:00.038561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:00.038619 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:00.038693 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:00.040171 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.040813 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.042194 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.042410 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.042489 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.044712 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:00.044715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:00.044717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:00.044741 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:00.045970 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.046392 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.046394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.046395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.046475 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.046722 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.046724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.046725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.046913 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:00.046989 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:00.047507 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 07:15:00.047602 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:00.048383 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.036774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44635 (local address 127.12.117.131:40157)
0810 07:15:00.037201 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:00.037240 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:00.038344 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:00.038865 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:00.038873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:00.039353 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:00.039971 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:00.039993 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.040940 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.040944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:00.042068 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:00.042080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.042521 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.042524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:00.042656 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.043145 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:00.043160 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:00.044544 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:00.046074 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:00.046079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:00.046089 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:00.046317 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:00.046542 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:00.046544 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:00.046545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:00.046658 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:00.047018 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:00.047024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:00.047308 (+ 284us) client_negotiation.cc:769] Sending connection context
0810 07:15:00.047461 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 07:15:00.047678 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260810 07:15:00.049396 13618 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44635
I20260810 07:15:00.049597 13618 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:00.050005 13618 heartbeater.cc:511] Master 127.12.117.190:44635 requested a full tablet report, sending...
I20260810 07:15:00.050905 12847 ts_manager.cc:194] Registered new tserver with Master: 2db5865a371f426b891e64d8078f969f (127.12.117.131:43197)
I20260810 07:15:00.051469 12847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:40157
I20260810 07:15:00.053791 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:00.078981 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.070023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58394 (local address 127.12.117.129:37565)
0810 07:15:00.070375 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:00.070386 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:00.070407 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:15:00.070459 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:00.070464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:00.070614 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:00.070736 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:00.071453 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.072235 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.073077 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.073422 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.073518 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.075737 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:00.075745 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:00.075756 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:00.075779 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:00.077466 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.077843 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.077848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.077849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.077939 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.078170 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.078172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.078173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.078378 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:00.078532 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:00.078669 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:15:00.078794 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":128,"threads_started":1}
W20260810 07:15:00.079861 13008 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58394
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[12772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[12772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:00.102686 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:00.095714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.12.117.129:37565)
0810 07:15:00.095854 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:00.095859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:00.095946 (+ 87us) server_negotiation.cc:407] Connection header received
0810 07:15:00.096041 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:00.096044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:00.096080 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:00.096142 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:00.096785 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.097284 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.098091 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:00.098299 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:00.098366 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:00.100359 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:00.100362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:00.100364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:00.100385 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:00.101447 (+ 1062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.101789 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.101792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.101793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.101880 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:00.102076 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:00.102078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:00.102079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:00.102219 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:00.102307 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:00.102464 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:15:00.102546 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:15:00.103895 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12933
I20260810 07:15:00.121527 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13161
I20260810 07:15:00.132321 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13402
I20260810 07:15:00.142073 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12791
2026-08-10T07:15:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2331 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[13636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[13636](info): set up 2 sockets
Aug 10 07:15:00 dist-test-slave-gjrz krb5kdc[13636](info): commencing operation
krb5kdc: starting...
W20260810 07:15:01.295527 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:01.409384 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34897
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:34897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:01.516482 13659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:01.516731 13659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:01.516786 13659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:01.520736 13659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:01.520787 13659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:01.520814 13659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:01.520836 13659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:01.520864 13659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:01.526003 13659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:34897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:01.526939 13659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:01.527906 13659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:01.535705 13669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.535715 13670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.535817 13659 server_base.cc:1034] running on GCE node
W20260810 07:15:01.535744 13673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.536477 13659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:01.537480 13659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:01.538643 13659 hybrid_clock.cc:648] HybridClock initialized: now 1786346101538615 us; error 38 us; skew 500 ppm
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:01.541230 13659 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:01.542062 13659 webserver.cc:460] Webserver started at http://127.12.117.190:40089/ using document root <none> and password file <none>
I20260810 07:15:01.542562 13659 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:01.542627 13659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:01.542806 13659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:01.544813 13659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "d8bac24a5f62433e94e3cb1862efaa99"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "ce9d6850d39f79359b69ab7daee50458"
server_key_iv: "cf8c4d9558284bf01cf5e840e42f669b"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.545341 13659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "d8bac24a5f62433e94e3cb1862efaa99"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "ce9d6850d39f79359b69ab7daee50458"
server_key_iv: "cf8c4d9558284bf01cf5e840e42f669b"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.548801 13659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:01.551002 13682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.552196 13659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:01.552333 13659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "d8bac24a5f62433e94e3cb1862efaa99"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "ce9d6850d39f79359b69ab7daee50458"
server_key_iv: "cf8c4d9558284bf01cf5e840e42f669b"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.552431 13659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:01.572417 13659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:01.574941 13659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:01.575107 13659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:01.581753 13659 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:34897
I20260810 07:15:01.581766 13766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:34897 every 8 connection(s)
I20260810 07:15:01.582633 13659 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:01.584547 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13659
I20260810 07:15:01.584645 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:01.584872 12758 external_mini_cluster.cc:1428] Setting key e4b7427af9b5531fb143815784cf2e72
I20260810 07:15:01.585379 13767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:01.591737 13767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: Bootstrap starting.
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:01.594038 13767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:01.594715 13767 log.cc:824] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:01.596457 13767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: No bootstrap required, opened a new log
I20260810 07:15:01.598291 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.586115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.12.117.190:34897)
0810 07:15:01.586516 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:01.586533 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:01.586568 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:01.587168 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:01.587184 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:01.587572 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:01.587838 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:01.588706 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.589673 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.590829 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.591316 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.591441 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.594004 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:01.594009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:01.594022 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:01.594049 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:01.595632 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.596088 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.596093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.596098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.596208 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.596487 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.596490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.596492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.596884 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:01.597058 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:01.597379 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 07:15:01.597647 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
I20260810 07:15:01.599933 13767 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } }
I20260810 07:15:01.600145 13767 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:01.600224 13767 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8bac24a5f62433e94e3cb1862efaa99, State: Initialized, Role: FOLLOWER
I20260810 07:15:01.600620 13767 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } }
I20260810 07:15:01.600745 13767 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:01.600808 13767 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:01.600900 13767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:01.601686 13767 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } }
I20260810 07:15:01.601999 13767 leader_election.cc:302] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8bac24a5f62433e94e3cb1862efaa99; no voters:
I20260810 07:15:01.602290 13767 leader_election.cc:288] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:01.602385 13775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:01.602852 13775 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [term 1 LEADER]: Becoming Leader. State: Replica: d8bac24a5f62433e94e3cb1862efaa99, State: Running, Role: LEADER
I20260810 07:15:01.603391 13775 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } }
I20260810 07:15:01.603451 13767 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:01.604969 13778 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8bac24a5f62433e94e3cb1862efaa99. Latest consensus state: current_term: 1 leader_uuid: "d8bac24a5f62433e94e3cb1862efaa99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } } }
I20260810 07:15:01.605002 13776 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8bac24a5f62433e94e3cb1862efaa99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8bac24a5f62433e94e3cb1862efaa99" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34897 } } }
I20260810 07:15:01.605101 13778 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:01.605100 13776 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:01.607802 13797 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:01.607875 13797 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:01.608049 13795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:01.608927 13795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:01.613816 13795 catalog_manager.cc:1347] Generated new cluster ID: 64a76464ac1b460aae59104d99ed1350
I20260810 07:15:01.613879 13795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:01.627312 13795 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:01.628134 13795 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:01.633234 13795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8bac24a5f62433e94e3cb1862efaa99: Generated new TSK 0
I20260810 07:15:01.633930 13795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:01.666065 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:01.773172 13807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:01.773406 13807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:01.773468 13807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:01.777431 13807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:01.777485 13807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:01.777565 13807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:01.782675 13807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:01.783622 13807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:01.784525 13807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:01.793452 13824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.793452 13815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.793475 13807 server_base.cc:1034] running on GCE node
W20260810 07:15:01.793509 13818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.794128 13807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:01.794732 13807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:01.795888 13807 hybrid_clock.cc:648] HybridClock initialized: now 1786346101795871 us; error 31 us; skew 500 ppm
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:01.798516 13807 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:01.799350 13807 webserver.cc:460] Webserver started at http://127.12.117.129:46515/ using document root <none> and password file <none>
I20260810 07:15:01.799865 13807 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:01.799930 13807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:01.800138 13807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:01.802150 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "74753997b73c4a39b1d216894e5a2dda"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "0a4af34dc0ddb2567843fa8383353487"
server_key_iv: "b4020b6dac9727684510150908bf76c0"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.802673 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "74753997b73c4a39b1d216894e5a2dda"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "0a4af34dc0ddb2567843fa8383353487"
server_key_iv: "b4020b6dac9727684510150908bf76c0"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.806293 13807 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:01.808661 13838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.809788 13807 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:01.809899 13807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "74753997b73c4a39b1d216894e5a2dda"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "0a4af34dc0ddb2567843fa8383353487"
server_key_iv: "b4020b6dac9727684510150908bf76c0"
server_key_version: "encryptionkey@0"
I20260810 07:15:01.809990 13807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:01.826989 13807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:01.829903 13807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:01.830083 13807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:01.830606 13807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:01.831416 13807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:01.831467 13807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.831518 13807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:01.831547 13807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:01.839619 13807 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:42345
I20260810 07:15:01.839648 14007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:42345 every 8 connection(s)
I20260810 07:15:01.840395 13807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:01.841943 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13807
I20260810 07:15:01.842039 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:01.842337 12758 external_mini_cluster.cc:1428] Setting key 2060d967eaf7987c5269d0a9a91f1ead
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:01.853421 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.842031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:34033 (local address 127.12.117.190:34897)
0810 07:15:01.842199 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:01.842204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:01.842973 (+ 769us) server_negotiation.cc:407] Connection header received
0810 07:15:01.844025 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:01.844028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:01.844089 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:01.844168 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:01.845263 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.845979 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.847104 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.847457 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.847571 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.850027 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:01.850032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:01.850035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:01.850064 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:01.851540 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.852017 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.852020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.852022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.852116 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:01.852410 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:01.852413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:01.852415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:01.852643 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:01.852751 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:01.853195 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 07:15:01.853303 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:15:01.854133 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:01.842366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34897 (local address 127.12.117.129:34033)
0810 07:15:01.842833 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:01.842874 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:01.843810 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:01.844332 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:01.844339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:01.844672 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:01.845117 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.845131 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.846124 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.846127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.846985 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:01.846995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:01.847585 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:01.847588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:01.847740 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:01.848235 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:01.848251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:01.849851 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:01.851660 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.851665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.851675 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.851914 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.852201 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:01.852203 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:01.852204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:01.852322 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:01.852736 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:01.852742 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:01.852974 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 07:15:01.853122 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 07:15:01.853352 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":117,"threads_started":1}
I20260810 07:15:01.855360 14008 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:15:01.855589 14008 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:01.856108 14008 heartbeater.cc:511] Master 127.12.117.190:34897 requested a full tablet report, sending...
I20260810 07:15:01.857551 13710 ts_manager.cc:194] Registered new tserver with Master: 74753997b73c4a39b1d216894e5a2dda (127.12.117.129:42345)
I20260810 07:15:01.858629 13710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:34033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:01.871518 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:01.979341 14017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:01.979564 14017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:01.979607 14017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:01.983560 14017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:01.983606 14017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:01.983675 14017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:01.988682 14017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:01.989595 14017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:01.990598 14017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:01.999414 14026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.999452 14034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:01.999414 14032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:01.999636 14017 server_base.cc:1034] running on GCE node
I20260810 07:15:02.000101 14017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:02.000689 14017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:02.001848 14017 hybrid_clock.cc:648] HybridClock initialized: now 1786346102001823 us; error 37 us; skew 500 ppm
Aug 10 07:15:01 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:02.004632 14017 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:02.005461 14017 webserver.cc:460] Webserver started at http://127.12.117.130:44305/ using document root <none> and password file <none>
I20260810 07:15:02.005962 14017 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:02.006021 14017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:02.006232 14017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:02.008296 14017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "6e8b95c8eebc4db797c40a7a30bc695e"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "1395034a533ef098158b6862e00f7423"
server_key_iv: "b5b8be4f662e79322a3202ee45f7400f"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.008818 14017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "6e8b95c8eebc4db797c40a7a30bc695e"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "1395034a533ef098158b6862e00f7423"
server_key_iv: "b5b8be4f662e79322a3202ee45f7400f"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.012275 14017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260810 07:15:02.014612 14041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.015756 14017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:02.015854 14017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "6e8b95c8eebc4db797c40a7a30bc695e"
format_stamp: "Formatted at 2026-08-10 07:15:01 on dist-test-slave-gjrz"
server_key: "1395034a533ef098158b6862e00f7423"
server_key_iv: "b5b8be4f662e79322a3202ee45f7400f"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.015949 14017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:02.033130 14017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:02.035665 14017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:02.035821 14017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:02.036347 14017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:02.037091 14017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:02.037137 14017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.037176 14017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:02.037205 14017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.045171 14017 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:36619
I20260810 07:15:02.045202 14233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:36619 every 8 connection(s)
I20260810 07:15:02.045953 14017 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:02.047359 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14017
I20260810 07:15:02.047461 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:02.047753 12758 external_mini_cluster.cc:1428] Setting key 39bf29607914dab23fa14248ca255e09
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:15:02.059535 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.047660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:42873 (local address 127.12.117.190:34897)
0810 07:15:02.047805 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.047810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.048619 (+ 809us) server_negotiation.cc:407] Connection header received
0810 07:15:02.049641 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.049644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.049697 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.049767 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.050884 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.051469 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.052761 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.053014 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.053101 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.055929 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.055932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.055934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.055958 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.057426 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.058035 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.058038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.058039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.058115 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.058511 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.058514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.058515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.058693 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.058769 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.059219 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.059367 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:15:02.060302 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.048015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34897 (local address 127.12.117.130:42873)
0810 07:15:02.048460 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.048514 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.049440 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.049929 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.049935 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.050266 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:02.050741 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.050756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.051589 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.051592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.052638 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.052651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.053170 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.053174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.053357 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.053912 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:02.053932 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:02.055733 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:02.057560 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.057567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.057583 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.057925 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.058200 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.058203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.058205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.058390 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.058793 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:02.058798 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:02.059023 (+ 225us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.059177 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.059554 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:15:02.061491 14234 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34897
I20260810 07:15:02.061743 14234 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:02.062311 14234 heartbeater.cc:511] Master 127.12.117.190:34897 requested a full tablet report, sending...
I20260810 07:15:02.063406 13710 ts_manager.cc:194] Registered new tserver with Master: 6e8b95c8eebc4db797c40a7a30bc695e (127.12.117.130:36619)
I20260810 07:15:02.063946 13710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:42873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:02.075385 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:02.183804 14241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:02.184048 14241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:02.184094 14241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:02.188093 14241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:02.188144 14241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:02.188200 14241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:02.193200 14241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34897
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:02.194090 14241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:02.195003 14241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:02.203938 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.203950 14259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:02.203950 14254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:02.204133 14241 server_base.cc:1034] running on GCE node
I20260810 07:15:02.204636 14241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:02.205206 14241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:02.206362 14241 hybrid_clock.cc:648] HybridClock initialized: now 1786346102206347 us; error 24 us; skew 500 ppm
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:02.208933 14241 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:02.209704 14241 webserver.cc:460] Webserver started at http://127.12.117.131:35817/ using document root <none> and password file <none>
I20260810 07:15:02.210210 14241 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:02.210269 14241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:02.210459 14241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:02.212373 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "9a735dc13c8744848e1557fb7d500077"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-gjrz"
server_key: "dd98e570d59c656845afc725f96beda9"
server_key_iv: "11e552d91ab5309293662d2f42892527"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.212896 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "9a735dc13c8744848e1557fb7d500077"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-gjrz"
server_key: "dd98e570d59c656845afc725f96beda9"
server_key_iv: "11e552d91ab5309293662d2f42892527"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.216398 14241 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:02.218614 14268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.219805 14241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:02.219910 14241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "9a735dc13c8744848e1557fb7d500077"
format_stamp: "Formatted at 2026-08-10 07:15:02 on dist-test-slave-gjrz"
server_key: "dd98e570d59c656845afc725f96beda9"
server_key_iv: "11e552d91ab5309293662d2f42892527"
server_key_version: "encryptionkey@0"
I20260810 07:15:02.220005 14241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:02.229166 14241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:02.231607 14241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:02.231763 14241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:02.232268 14241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:02.232983 14241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:02.233031 14241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.233088 14241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:02.233119 14241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:02.240851 14241 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:42729
I20260810 07:15:02.240908 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:42729 every 8 connection(s)
I20260810 07:15:02.241626 14241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:02.250669 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14241
I20260810 07:15:02.250784 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:02.251046 12758 external_mini_cluster.cc:1428] Setting key f7b2cf5affb64f426f85ed0fd341c783
I20260810 07:15:02.253551 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.243250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:33909 (local address 127.12.117.190:34897)
0810 07:15:02.243401 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.243406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.244188 (+ 782us) server_negotiation.cc:407] Connection header received
0810 07:15:02.244969 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.244972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.245017 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.245088 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.246104 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.246614 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.247772 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.248006 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.248097 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.250179 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.250182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.250185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.250209 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.251591 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.252090 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.252094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.252095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.252187 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.252531 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.252534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.252535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.252702 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.252778 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.253340 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.253440 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:15:02.254261 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.243616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34897 (local address 127.12.117.131:33909)
0810 07:15:02.244032 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.244072 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.244774 (+ 702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.245232 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.245239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.245546 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:02.245984 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.245996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.246703 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.246705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.247640 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.247650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.248126 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.248129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.248262 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.248700 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:02.248714 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:02.250015 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:02.251701 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.251707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.251720 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.252000 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.252277 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:02.252280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:02.252282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:02.252437 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:02.252800 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:02.252805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:02.253098 (+ 293us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.253281 (+ 183us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.253507 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":112,"threads_started":1}
I20260810 07:15:02.255321 14454 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34897
I20260810 07:15:02.255530 14454 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:02.255968 14454 heartbeater.cc:511] Master 127.12.117.190:34897 requested a full tablet report, sending...
I20260810 07:15:02.256896 13710 ts_manager.cc:194] Registered new tserver with Master: 9a735dc13c8744848e1557fb7d500077 (127.12.117.131:42729)
I20260810 07:15:02.257402 13710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:33909
I20260810 07:15:02.264592 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:02.273617 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.267625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.12.117.190:34897)
0810 07:15:02.267732 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.267736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.267875 (+ 139us) server_negotiation.cc:407] Connection header received
0810 07:15:02.267957 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.267959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.267991 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.268052 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.268602 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.269062 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.269770 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.269962 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.270025 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.271073 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.271076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.271078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.271098 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.272338 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.272705 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.272707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.272708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.272790 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.272987 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.272989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.272990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.273171 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.273267 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.273421 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.273505 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:15:02.277685 13710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:02.280629 13710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:02.294315 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:42729 (local address 127.0.0.1:39312)
0810 07:15:02.288975 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.289008 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.289230 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.289896 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.289901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.289923 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:02.290040 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.290047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.291497 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.291500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.292797 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.292809 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.293793 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.293797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.293901 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.293940 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.294062 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.294182 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":41,"thread_start_us":61,"threads_started":1}
I20260810 07:15:02.294345 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.12.117.131:42729)
0810 07:15:02.288614 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.288623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.289102 (+ 479us) server_negotiation.cc:407] Connection header received
0810 07:15:02.289391 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.289396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.289600 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.289737 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:02.290167 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.291366 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.292927 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.293671 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.293791 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.293916 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.294118 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.294208 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260810 07:15:02.294513 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36619 (local address 127.0.0.1:44346)
0810 07:15:02.289446 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.289458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.289532 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.290021 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.290023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.290039 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:02.290144 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.290150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.292428 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.292432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.293258 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.293268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.294135 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.294139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.294229 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.294252 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.294323 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.294392 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260810 07:15:02.294827 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44346 (local address 127.12.117.130:36619)
0810 07:15:02.289229 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.289240 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.289619 (+ 379us) server_negotiation.cc:407] Connection header received
0810 07:15:02.289699 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.289705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.289880 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.290027 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:02.290251 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.291368 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.293381 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.294020 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.294130 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.294243 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.294600 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.294691 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":126,"threads_started":1}
I20260810 07:15:02.295127 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.12.117.129:42345)
0810 07:15:02.289325 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.289337 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.289755 (+ 418us) server_negotiation.cc:407] Connection header received
0810 07:15:02.289839 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.289844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.290046 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.290192 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:02.290469 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.291182 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.292812 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.293666 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.293791 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.293916 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.294893 (+ 977us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.294993 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":97,"threads_started":1}
I20260810 07:15:02.294704 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.288675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42345 (local address 127.0.0.1:35694)
0810 07:15:02.289561 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.289575 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.289662 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.290267 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.290269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.290284 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:02.290362 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.290368 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.291309 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.291312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.292683 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.292693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.294318 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.294322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.294421 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.294445 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.294521 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.294584 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260810 07:15:02.297560 14121 tablet_service.cc:1467] Processing CreateTablet for tablet e4b6edcb4fb44440b9ccc150ad9461b1 (DEFAULT_TABLE table=test-table [id=6643d811a64b4bb49a8b07db4d1f62e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:02.297591 14344 tablet_service.cc:1467] Processing CreateTablet for tablet e4b6edcb4fb44440b9ccc150ad9461b1 (DEFAULT_TABLE table=test-table [id=6643d811a64b4bb49a8b07db4d1f62e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:02.297775 13909 tablet_service.cc:1467] Processing CreateTablet for tablet e4b6edcb4fb44440b9ccc150ad9461b1 (DEFAULT_TABLE table=test-table [id=6643d811a64b4bb49a8b07db4d1f62e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:02.298492 14121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b6edcb4fb44440b9ccc150ad9461b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:02.298576 13909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b6edcb4fb44440b9ccc150ad9461b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:02.298584 14344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4b6edcb4fb44440b9ccc150ad9461b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:02.305574 14489 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Bootstrap starting.
I20260810 07:15:02.306114 14490 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Bootstrap starting.
I20260810 07:15:02.306114 14488 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: Bootstrap starting.
I20260810 07:15:02.307979 14489 tablet_bootstrap.cc:654] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:02.308207 14490 tablet_bootstrap.cc:654] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:02.308579 14488 tablet_bootstrap.cc:654] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:02.309204 14489 log.cc:824] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:02.309412 14490 log.cc:824] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:02.309412 14488 log.cc:824] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:02.311093 14488 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: No bootstrap required, opened a new log
I20260810 07:15:02.311300 14488 ts_tablet_manager.cc:1397] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:15:02.311353 14489 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: No bootstrap required, opened a new log
I20260810 07:15:02.311399 14490 tablet_bootstrap.cc:492] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: No bootstrap required, opened a new log
I20260810 07:15:02.311540 14490 ts_tablet_manager.cc:1397] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 07:15:02.311540 14489 ts_tablet_manager.cc:1397] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:15:02.314160 14488 raft_consensus.cc:348] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.314357 14488 raft_consensus.cc:374] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:02.314386 14488 raft_consensus.cc:729] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e8b95c8eebc4db797c40a7a30bc695e, State: Initialized, Role: FOLLOWER
I20260810 07:15:02.314771 14488 consensus_queue.cc:260] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.315347 14234 heartbeater.cc:503] Master 127.12.117.190:34897 was elected leader, sending a full tablet report...
I20260810 07:15:02.315255 14489 raft_consensus.cc:348] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.315594 14489 raft_consensus.cc:374] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:02.315650 14489 raft_consensus.cc:729] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74753997b73c4a39b1d216894e5a2dda, State: Initialized, Role: FOLLOWER
I20260810 07:15:02.315675 14488 ts_tablet_manager.cc:1428] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:15:02.315255 14490 raft_consensus.cc:348] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.315750 14490 raft_consensus.cc:374] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:02.315793 14490 raft_consensus.cc:729] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a735dc13c8744848e1557fb7d500077, State: Initialized, Role: FOLLOWER
I20260810 07:15:02.316232 14490 consensus_queue.cc:260] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.316443 14489 consensus_queue.cc:260] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.317109 14454 heartbeater.cc:503] Master 127.12.117.190:34897 was elected leader, sending a full tablet report...
I20260810 07:15:02.317230 14490 ts_tablet_manager.cc:1428] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 07:15:02.317282 14489 ts_tablet_manager.cc:1428] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:15:02.317548 14008 heartbeater.cc:503] Master 127.12.117.190:34897 was elected leader, sending a full tablet report...
W20260810 07:15:02.341375 14009 tablet.cc:2369] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:02.379719 14495 raft_consensus.cc:482] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:02.379949 14495 raft_consensus.cc:504] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.381104 14495 leader_election.cc:288] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e8b95c8eebc4db797c40a7a30bc695e (127.12.117.130:36619), 74753997b73c4a39b1d216894e5a2dda (127.12.117.129:42345)
I20260810 07:15:02.385175 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.381359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36619 (local address 127.12.117.131:53561)
0810 07:15:02.381470 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.381484 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.381568 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.381993 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.381995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.382013 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:02.382111 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.382116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.382970 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.382974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.383915 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.383925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.384728 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.384732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.384845 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.384873 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.384957 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.385038 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:15:02.385175 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.381499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:53561 (local address 127.12.117.130:36619)
0810 07:15:02.381623 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.381628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.381642 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:02.381693 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.381696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.381745 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.381820 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:02.382213 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.382859 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.384048 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.384596 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.384686 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.384728 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.384979 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.385051 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:15:02.385432 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.381608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42345 (local address 127.12.117.131:42531)
0810 07:15:02.382021 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:02.382033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:02.382135 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:02.382406 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:02.382410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:02.382427 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:02.382554 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.382560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.383415 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.383417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.384283 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:02.384294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.385038 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.385042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:02.385146 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.385170 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:15:02.385247 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:15:02.385316 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":115,"threads_started":1}
I20260810 07:15:02.385587 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.381796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:42531 (local address 127.12.117.129:42345)
0810 07:15:02.381958 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.381962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.382119 (+ 157us) server_negotiation.cc:407] Connection header received
0810 07:15:02.382269 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.382271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.382314 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.382371 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:02.382649 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.383313 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.384425 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.384891 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.385035 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.385081 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.385393 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.385475 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 07:15:02.386086 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b6edcb4fb44440b9ccc150ad9461b1" candidate_uuid: "9a735dc13c8744848e1557fb7d500077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74753997b73c4a39b1d216894e5a2dda" is_pre_election: true
I20260810 07:15:02.386085 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b6edcb4fb44440b9ccc150ad9461b1" candidate_uuid: "9a735dc13c8744848e1557fb7d500077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" is_pre_election: true
I20260810 07:15:02.386345 13938 raft_consensus.cc:2454] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a735dc13c8744848e1557fb7d500077 in term 0.
I20260810 07:15:02.386345 14155 raft_consensus.cc:2454] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a735dc13c8744848e1557fb7d500077 in term 0.
I20260810 07:15:02.386780 14271 leader_election.cc:302] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74753997b73c4a39b1d216894e5a2dda, 9a735dc13c8744848e1557fb7d500077; no voters:
I20260810 07:15:02.387022 14495 raft_consensus.cc:2781] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:02.387115 14495 raft_consensus.cc:482] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:02.387166 14495 raft_consensus.cc:3037] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:02.388164 14495 raft_consensus.cc:504] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.388445 14495 leader_election.cc:288] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [CANDIDATE]: Term 1 election: Requested vote from peers 6e8b95c8eebc4db797c40a7a30bc695e (127.12.117.130:36619), 74753997b73c4a39b1d216894e5a2dda (127.12.117.129:42345)
I20260810 07:15:02.388778 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b6edcb4fb44440b9ccc150ad9461b1" candidate_uuid: "9a735dc13c8744848e1557fb7d500077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e8b95c8eebc4db797c40a7a30bc695e"
I20260810 07:15:02.388839 13938 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4b6edcb4fb44440b9ccc150ad9461b1" candidate_uuid: "9a735dc13c8744848e1557fb7d500077" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74753997b73c4a39b1d216894e5a2dda"
I20260810 07:15:02.388885 14155 raft_consensus.cc:3037] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:02.388947 13938 raft_consensus.cc:3037] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:02.389863 14155 raft_consensus.cc:2454] T e4b6edcb4fb44440b9ccc150ad9461b1 P 6e8b95c8eebc4db797c40a7a30bc695e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a735dc13c8744848e1557fb7d500077 in term 1.
I20260810 07:15:02.389941 13938 raft_consensus.cc:2454] T e4b6edcb4fb44440b9ccc150ad9461b1 P 74753997b73c4a39b1d216894e5a2dda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a735dc13c8744848e1557fb7d500077 in term 1.
I20260810 07:15:02.390125 14271 leader_election.cc:302] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e8b95c8eebc4db797c40a7a30bc695e, 9a735dc13c8744848e1557fb7d500077; no voters:
I20260810 07:15:02.390281 14495 raft_consensus.cc:2781] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:02.390488 14495 raft_consensus.cc:686] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [term 1 LEADER]: Becoming Leader. State: Replica: 9a735dc13c8744848e1557fb7d500077, State: Running, Role: LEADER
I20260810 07:15:02.390792 14495 consensus_queue.cc:237] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } }
I20260810 07:15:02.393888 13710 catalog_manager.cc:5495] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a735dc13c8744848e1557fb7d500077 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "9a735dc13c8744848e1557fb7d500077" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a735dc13c8744848e1557fb7d500077" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e8b95c8eebc4db797c40a7a30bc695e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74753997b73c4a39b1d216894e5a2dda" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42345 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:02.414763 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.406579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.12.117.129:42345)
0810 07:15:02.406688 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.406691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.406839 (+ 148us) server_negotiation.cc:407] Connection header received
0810 07:15:02.406954 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.406957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.407011 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.407078 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.407788 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.408253 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.409077 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.409288 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.409374 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.411679 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.411689 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.411701 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.411723 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.413088 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.413515 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.413520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.413521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.413608 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.413886 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.413889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.413891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.414137 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.414355 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.414526 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.414637 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:02.440011 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.432666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35724 (local address 127.12.117.129:42345)
0810 07:15:02.432801 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.432804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.432887 (+ 83us) server_negotiation.cc:407] Connection header received
0810 07:15:02.432981 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.432983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.433019 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.433080 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.433641 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.434106 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.434836 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.435044 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.435122 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.437193 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.437196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.437198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.437219 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.438432 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.438809 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.438812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.438814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.438886 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.439160 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.439163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.439165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.439364 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.439488 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.439748 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.439854 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[13636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:02.463011 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:02.455865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.12.117.129:42345)
0810 07:15:02.455988 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:02.455992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:02.456173 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:15:02.456259 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:02.456261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:02.456299 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:02.456364 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:02.456936 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.457412 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.458186 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:02.458398 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:02.458493 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:02.460576 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:02.460580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:02.460581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:02.460605 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:02.461739 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.462117 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.462119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.462120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.462186 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:02.462373 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:02.462375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:02.462376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:02.462523 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:02.462611 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:02.462772 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 07:15:02.462858 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:15:02.464495 13881 tablet_service.cc:2908] Rejecting scan request for tablet e4b6edcb4fb44440b9ccc150ad9461b1: Uninitialized: clean time has not yet been initialized
I20260810 07:15:02.466486 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13807
I20260810 07:15:02.484606 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14017
W20260810 07:15:02.492568 14458 tablet.cc:2369] T e4b6edcb4fb44440b9ccc150ad9461b1 P 9a735dc13c8744848e1557fb7d500077: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:02.525734 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14241
I20260810 07:15:02.537914 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13659
2026-08-10T07:15:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2394 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[14520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[14520](info): set up 2 sockets
Aug 10 07:15:02 dist-test-slave-gjrz krb5kdc[14520](info): commencing operation
krb5kdc: starting...
W20260810 07:15:03.711413 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:03 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:03.823988 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:40575
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:40575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:03.929466 14536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:03.929677 14536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:03.929718 14536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:03.933558 14536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:03.933600 14536 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:15:03.933616 14536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:03.933647 14536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:03.933676 14536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:03.933699 14536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:03.938619 14536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:40575
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:40575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:03.939463 14536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:03.940335 14536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:03.948081 14546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.948088 14545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:03.948201 14549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:03.948398 14536 server_base.cc:1034] running on GCE node
I20260810 07:15:03.949007 14536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:03.949927 14536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:03.951092 14536 hybrid_clock.cc:648] HybridClock initialized: now 1786346103951079 us; error 25 us; skew 500 ppm
Aug 10 07:15:03 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:03.960798 14536 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:03.961577 14536 webserver.cc:460] Webserver started at http://127.12.117.190:32915/ using document root <none> and password file <none>
I20260810 07:15:03.962052 14536 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:03.962108 14536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:03.962265 14536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:03.964134 14536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2fed4831a5d44d118cce0102787df6d8"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-gjrz"
server_key: "ef1f3aad5d850f262f8474f1ce9f5bb8"
server_key_iv: "52b33d134716e3340f95394a93fb3110"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.964613 14536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2fed4831a5d44d118cce0102787df6d8"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-gjrz"
server_key: "ef1f3aad5d850f262f8474f1ce9f5bb8"
server_key_iv: "52b33d134716e3340f95394a93fb3110"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.967941 14536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:03.970155 14565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:03.971231 14536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:03.971362 14536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2fed4831a5d44d118cce0102787df6d8"
format_stamp: "Formatted at 2026-08-10 07:15:03 on dist-test-slave-gjrz"
server_key: "ef1f3aad5d850f262f8474f1ce9f5bb8"
server_key_iv: "52b33d134716e3340f95394a93fb3110"
server_key_version: "encryptionkey@0"
I20260810 07:15:03.971444 14536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:03.986641 14536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:03.989348 14536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:03.989485 14536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:03.996691 14536 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:40575
I20260810 07:15:03.996701 14675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:40575 every 8 connection(s)
I20260810 07:15:03.997515 14536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:03.998822 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14536
I20260810 07:15:03.998909 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:03.999126 12758 external_mini_cluster.cc:1428] Setting key c535108777af250c05ae5edbe4b57192
I20260810 07:15:04.000262 14676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.006639 14676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Bootstrap starting.
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:04.009073 14676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.010456 14676 log.cc:824] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:04.012602 14676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: No bootstrap required, opened a new log
I20260810 07:15:04.013504 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.000402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.12.117.190:40575)
0810 07:15:04.000813 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.000830 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.000868 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:15:04.001658 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.001681 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.002095 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.002474 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.003097 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.004596 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.005441 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.006119 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.006269 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.008733 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.008739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.008754 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.008785 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.010648 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.011130 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.011134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.011138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.011282 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.011529 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.011532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.011534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.012002 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.012178 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.012362 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.012643 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":124,"threads_started":1}
I20260810 07:15:04.015933 14676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } }
I20260810 07:15:04.016125 14676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.016170 14676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fed4831a5d44d118cce0102787df6d8, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.016532 14676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } }
I20260810 07:15:04.016649 14676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:04.016692 14676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:04.016765 14676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:04.017699 14676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } }
I20260810 07:15:04.017974 14676 leader_election.cc:302] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fed4831a5d44d118cce0102787df6d8; no voters:
I20260810 07:15:04.018198 14676 leader_election.cc:288] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:04.018316 14686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:04.018508 14686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 2fed4831a5d44d118cce0102787df6d8, State: Running, Role: LEADER
I20260810 07:15:04.018795 14686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } }
I20260810 07:15:04.019220 14676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:04.020294 14689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fed4831a5d44d118cce0102787df6d8. Latest consensus state: current_term: 1 leader_uuid: "2fed4831a5d44d118cce0102787df6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } } }
I20260810 07:15:04.020494 14689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:04.020823 14687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fed4831a5d44d118cce0102787df6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fed4831a5d44d118cce0102787df6d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 40575 } } }
I20260810 07:15:04.020897 14700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:04.023324 14687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:04.024012 14700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:04.028733 14700 catalog_manager.cc:1347] Generated new cluster ID: 1a73af5db7bc45ad8798785db859cb2b
I20260810 07:15:04.028795 14700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:04.059695 14700 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:04.060617 14700 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:04.069543 14700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Generated new TSK 0
I20260810 07:15:04.070147 14700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:04.098523 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:40575
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:04.206327 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:04.206537 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:04.206581 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:04.210438 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:04.210486 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:04.210530 14721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:04.210580 14721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:04.215505 14721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:04.216351 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:04.217242 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:04.226281 14727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.226254 14728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.226261 14731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.226416 14721 server_base.cc:1034] running on GCE node
I20260810 07:15:04.226876 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:04.227464 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:04.228611 14721 hybrid_clock.cc:648] HybridClock initialized: now 1786346104228589 us; error 35 us; skew 500 ppm
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:04.231066 14721 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:04.231866 14721 webserver.cc:460] Webserver started at http://127.12.117.129:34535/ using document root <none> and password file <none>
I20260810 07:15:04.232363 14721 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:04.232426 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:04.232606 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:04.234447 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "7261b1caf4d14cdca5df9b8cc0e57c15"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "5151eff37b370a9ddbbbc4cbc8f946e1"
server_key_iv: "ee5d9a0def7a02222b2ab513cd1d02f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.234961 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "7261b1caf4d14cdca5df9b8cc0e57c15"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "5151eff37b370a9ddbbbc4cbc8f946e1"
server_key_iv: "ee5d9a0def7a02222b2ab513cd1d02f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.238572 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:04.240808 14740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.241922 14721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:04.242020 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "7261b1caf4d14cdca5df9b8cc0e57c15"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "5151eff37b370a9ddbbbc4cbc8f946e1"
server_key_iv: "ee5d9a0def7a02222b2ab513cd1d02f1"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.242110 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:04.267467 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:04.269790 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:04.269944 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:04.271198 14721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:04.271243 14721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.271310 14721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:04.271337 14721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:04.280959 14721 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:37671
I20260810 07:15:04.281041 14949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:37671 every 8 connection(s)
I20260810 07:15:04.281915 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:04.283648 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14721
I20260810 07:15:04.283757 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:04.284044 12758 external_mini_cluster.cc:1428] Setting key 7b7bc5d9511d20b7f191eee1e2d36ccb
I20260810 07:15:04.284301 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.271902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:54375 (local address 127.12.117.190:40575)
0810 07:15:04.272078 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.272083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.273164 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 07:15:04.274249 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.274252 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.274308 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.274387 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.275920 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.276455 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.277682 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.277914 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.278002 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.280972 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.280976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.280979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.281020 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.282346 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.282849 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.282852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.282853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.282934 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.283174 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.283175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.283177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.283404 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.283482 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.284051 (+ 569us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.284184 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:15:04.285082 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.272601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.129:54375)
0810 07:15:04.273017 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.273064 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.274051 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.274582 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.274591 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.275034 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.275736 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.275757 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.276579 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.276583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.277565 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.277576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.278061 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.278065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.278216 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.279412 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.279431 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.280806 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.282472 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.282477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.282489 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.282756 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.282999 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.283001 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.283002 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.283120 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.283507 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.283513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.283790 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.283990 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.284295 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":102,"threads_started":1}
I20260810 07:15:04.286185 14950 heartbeater.cc:347] Connected to a master server at 127.12.117.190:40575
I20260810 07:15:04.286309 14950 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:04.286697 14950 heartbeater.cc:511] Master 127.12.117.190:40575 requested a full tablet report, sending...
I20260810 07:15:04.288255 14592 ts_manager.cc:194] Registered new tserver with Master: 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129:37671)
I20260810 07:15:04.289727 14592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:54375
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:04.295512 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.288437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:43073 (local address 127.12.117.190:40575)
0810 07:15:04.288552 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.288556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.288863 (+ 307us) server_negotiation.cc:407] Connection header received
0810 07:15:04.289046 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.289049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.289090 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.289137 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.289704 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.290379 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.291452 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.291796 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.291902 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.292830 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.292835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.292837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.292864 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.294214 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.294605 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.294609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.294611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.294694 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.294884 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.294885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.294887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.295030 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.295095 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.295301 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.295402 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:15:04.295528 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.288517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.129:43073)
0810 07:15:04.288785 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.288800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.288928 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.289259 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.289262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.289454 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.289566 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.289573 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.290475 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.290478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.291331 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.291342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.291917 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.291920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.291989 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.292400 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.292410 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.292693 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.294314 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.294317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.294318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.294513 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.294763 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.294765 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.294766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.294830 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.295089 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.295091 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.295159 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.295233 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.295378 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:04.310107 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:40575
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:04.416159 14968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:04.416442 14968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:04.416502 14968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:04.420491 14968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:04.420542 14968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:04.420593 14968 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:04.420646 14968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:04.425683 14968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:04.426601 14968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:04.427672 14968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:04.436774 14974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.436811 14968 server_base.cc:1034] running on GCE node
W20260810 07:15:04.436772 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.437103 14977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.437449 14968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:04.438042 14968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:04.439209 14968 hybrid_clock.cc:648] HybridClock initialized: now 1786346104439192 us; error 27 us; skew 500 ppm
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:04.441663 14968 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:04.442463 14968 webserver.cc:460] Webserver started at http://127.12.117.130:43913/ using document root <none> and password file <none>
I20260810 07:15:04.442968 14968 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:04.443032 14968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:04.443222 14968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:04.445205 14968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "cbf273a273de4e919f406c4e6b92388a"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9ec774286b4c3fd14c119c8210d1a8cc"
server_key_iv: "7bceba479c60e41e1d026ce002aa3df9"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.445741 14968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "cbf273a273de4e919f406c4e6b92388a"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9ec774286b4c3fd14c119c8210d1a8cc"
server_key_iv: "7bceba479c60e41e1d026ce002aa3df9"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.449222 14968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:04.451427 14986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.452587 14968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:04.452693 14968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "cbf273a273de4e919f406c4e6b92388a"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9ec774286b4c3fd14c119c8210d1a8cc"
server_key_iv: "7bceba479c60e41e1d026ce002aa3df9"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.452780 14968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:04.470685 14968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:04.473091 14968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:04.473245 14968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:04.474540 14968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:04.474588 14968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.474637 14968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:04.474665 14968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:04.484691 14968 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:39889
I20260810 07:15:04.484735 15181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:39889 every 8 connection(s)
I20260810 07:15:04.485536 14968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:04.487309 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.475431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:41311 (local address 127.12.117.190:40575)
0810 07:15:04.475540 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.475543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.476546 (+ 1003us) server_negotiation.cc:407] Connection header received
0810 07:15:04.477519 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.477523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.477592 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.477651 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.479220 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.479844 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.481044 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.481295 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.481389 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.483828 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.483832 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.483835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.483865 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.485337 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.485866 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.485870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.485872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.485966 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.486373 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.486376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.486378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.486580 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.486663 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.487062 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.487161 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:04.487983 15000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.475958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.130:41311)
0810 07:15:04.476407 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.476445 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.477271 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.478055 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.478065 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.478442 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.479077 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.479097 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.479961 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.479965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.480894 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.480907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.481454 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.481458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.481652 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.482177 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.482194 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.483671 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.485449 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.485456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.485470 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.485772 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.486085 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.486088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.486090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.486244 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.486658 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.486663 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.486865 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.487008 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.487238 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":122,"threads_started":1}
I20260810 07:15:04.489085 15182 heartbeater.cc:347] Connected to a master server at 127.12.117.190:40575
I20260810 07:15:04.489235 15182 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:04.489720 15182 heartbeater.cc:511] Master 127.12.117.190:40575 requested a full tablet report, sending...
I20260810 07:15:04.490676 14596 ts_manager.cc:194] Registered new tserver with Master: cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889)
I20260810 07:15:04.491428 14596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:41311
I20260810 07:15:04.495378 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14968
I20260810 07:15:04.495491 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:04.495774 12758 external_mini_cluster.cc:1428] Setting key b4ed5e02416615fb663bb6a83afb82e6
I20260810 07:15:04.497121 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.491009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.130:42037)
0810 07:15:04.491368 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.491379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.491466 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.491666 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.491669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.491842 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.491929 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.491935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.492642 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.492645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.493185 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.493193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.493600 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.493604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.493692 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.494061 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.494071 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.494333 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.495605 (+ 1272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.495609 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.495611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.495842 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.496095 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.496098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.496100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.496194 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.496645 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.496648 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.496735 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.496841 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.496969 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":88,"threads_started":1}
I20260810 07:15:04.497136 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.491085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:42037 (local address 127.12.117.190:40575)
0810 07:15:04.491203 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.491207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.491443 (+ 236us) server_negotiation.cc:407] Connection header received
0810 07:15:04.491536 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.491539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.491580 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.491642 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.492034 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.492522 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.493290 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.493501 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.493586 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.494456 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.494460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.494462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.494486 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.495508 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.495937 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.495940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.495941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.496014 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.496293 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.496296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.496298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.496538 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.496623 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.496902 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.497005 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:04.520190 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:40575
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:04.626292 15202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:04.626499 15202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:04.626538 15202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:04.630316 15202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:04.630360 15202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:04.630395 15202 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:04.630424 15202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:04.635332 15202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:40575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:04.636139 15202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:04.636974 15202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:04.645586 15209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.645586 15212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:04.645596 15210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:04.645875 15202 server_base.cc:1034] running on GCE node
I20260810 07:15:04.646274 15202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:04.646827 15202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:04.647980 15202 hybrid_clock.cc:648] HybridClock initialized: now 1786346104647961 us; error 30 us; skew 500 ppm
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:04.650508 15202 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:04.651331 15202 webserver.cc:460] Webserver started at http://127.12.117.131:45383/ using document root <none> and password file <none>
I20260810 07:15:04.651829 15202 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:04.651896 15202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:04.652127 15202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:04.653977 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "1d59021628d7432984a1e69f57037163"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9f6d5864e252e352a8d64bb85920ad1c"
server_key_iv: "e491e84bae4522b69a7eaab166b55530"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.654508 15202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "1d59021628d7432984a1e69f57037163"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9f6d5864e252e352a8d64bb85920ad1c"
server_key_iv: "e491e84bae4522b69a7eaab166b55530"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.658094 15202 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:04.660378 15223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.661494 15202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:04.661598 15202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "1d59021628d7432984a1e69f57037163"
format_stamp: "Formatted at 2026-08-10 07:15:04 on dist-test-slave-gjrz"
server_key: "9f6d5864e252e352a8d64bb85920ad1c"
server_key_iv: "e491e84bae4522b69a7eaab166b55530"
server_key_version: "encryptionkey@0"
I20260810 07:15:04.661689 15202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:04.670313 15202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:04.673158 15202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:04.673317 15202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:04.674654 15202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:04.674702 15202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:04.674751 15202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:04.674782 15202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:04.684027 15202 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41489
I20260810 07:15:04.684156 15411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41489 every 8 connection(s)
I20260810 07:15:04.685007 15202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:04.685420 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15202
I20260810 07:15:04.685535 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:04.685807 12758 external_mini_cluster.cc:1428] Setting key b547724ec878c97882fc6192730a8736
I20260810 07:15:04.688140 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.675258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:37619 (local address 127.12.117.190:40575)
0810 07:15:04.675415 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.675421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.676632 (+ 1211us) server_negotiation.cc:407] Connection header received
0810 07:15:04.677745 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.677749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.677809 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.677893 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.679427 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.680213 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.681354 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.681594 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.681685 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.684297 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.684301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.684304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.684335 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.685867 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.686445 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.686448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.686449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.686539 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.686953 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.686957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.686959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.687195 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.687303 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.687860 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.687977 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:15:04.688851 15242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.676066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.131:37619)
0810 07:15:04.676484 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.676530 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.677511 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.678112 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.678122 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.678583 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.679231 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.679251 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.680343 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.680346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.681227 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.681238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.681735 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.681740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.681926 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.682471 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.682489 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.684111 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.686068 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.686075 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.686089 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.686338 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.686715 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.686717 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.686719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.686849 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.687316 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.687322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.687610 (+ 288us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.687798 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.688086 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":97,"threads_started":1}
I20260810 07:15:04.689808 15416 heartbeater.cc:347] Connected to a master server at 127.12.117.190:40575
I20260810 07:15:04.689951 15416 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:04.690409 15416 heartbeater.cc:511] Master 127.12.117.190:40575 requested a full tablet report, sending...
I20260810 07:15:04.691342 14592 ts_manager.cc:194] Registered new tserver with Master: 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:04.691916 14592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:37619
I20260810 07:15:04.697572 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.691882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:49927 (local address 127.12.117.190:40575)
0810 07:15:04.691997 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.692001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.692190 (+ 189us) server_negotiation.cc:407] Connection header received
0810 07:15:04.692291 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.692294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.692331 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.692386 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.692806 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.693524 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.694245 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.694445 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.694518 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.695416 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.695418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.695420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.695438 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.696427 (+ 989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.696764 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.696766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.696767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.696829 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.697010 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.697012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.697013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.697151 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.697217 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.697396 (+ 179us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.697475 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:04.697575 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.691800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.12.117.131:49927)
0810 07:15:04.692130 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.692141 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.692220 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.692415 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.692418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.692607 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:04.692700 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.692706 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.693629 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.693631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.694154 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.694161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.694532 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.694534 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.694600 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.694984 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:04.694997 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:04.695310 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:04.696515 (+ 1205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.696517 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.696518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.696696 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.696887 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:04.696888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:04.696890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:04.696951 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:04.697213 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:04.697214 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:04.697283 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.697355 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.697448 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":91,"threads_started":1}
I20260810 07:15:04.699395 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:04 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:04.723439 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.716252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55852 (local address 127.12.117.190:40575)
0810 07:15:04.716364 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.716367 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.716381 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:04.716438 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.716440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.716476 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.716540 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:04.717197 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.717672 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.718439 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.718649 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.718723 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.720898 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:04.720901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:04.720903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:04.720924 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:04.722151 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.722542 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.722544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.722545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.722609 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:04.722794 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:04.722796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:04.722797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:04.722969 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:04.723069 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.723215 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.723331 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:15:04.725837 14592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:04.728735 14592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:04.742585 15439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.736546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:39889 (local address 127.0.0.1:55982)
0810 07:15:04.737295 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.737342 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.737464 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.738334 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.738340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.738377 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:04.738526 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.738533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.739641 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.739644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.741212 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.741220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.742074 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.742185 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742223 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.742324 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.742450 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":99,"spinlock_wait_cycles":7936,"thread_start_us":70,"threads_started":1}
I20260810 07:15:04.742594 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.736697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.12.117.130:39889)
0810 07:15:04.737840 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.737851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.737870 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:15:04.737935 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.737939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.738079 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.738194 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:04.738656 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.739534 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.741305 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.741955 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742088 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742220 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.742361 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.742449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":113,"threads_started":1}
I20260810 07:15:04.742939 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.736672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37671 (local address 127.0.0.1:52802)
0810 07:15:04.737513 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.737527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.737616 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.738334 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.738340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.738375 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:04.738529 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.738534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.739894 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.739897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.741139 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.741151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.742448 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.742605 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742645 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.742727 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.742793 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":99,"spinlock_wait_cycles":7808,"thread_start_us":79,"threads_started":1}
I20260810 07:15:04.743343 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.737178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.12.117.131:41489)
0810 07:15:04.737452 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.737464 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.737936 (+ 472us) server_negotiation.cc:407] Connection header received
0810 07:15:04.738155 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.738162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.738404 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.738515 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:04.738921 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.740000 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.741351 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.742126 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742257 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742372 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.743092 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.743187 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
I20260810 07:15:04.743462 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.737331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.12.117.129:37671)
0810 07:15:04.737677 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.737689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.737713 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:15:04.737787 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.737792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.738011 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.738169 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:04.738678 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.739780 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.741281 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.742126 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742254 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742372 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.743209 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.743329 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":144,"threads_started":1}
I20260810 07:15:04.743706 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.736955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41489 (local address 127.0.0.1:60880)
0810 07:15:04.737844 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.737857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.738006 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.738655 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.738658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.738676 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:04.738786 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.738792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.740109 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.740112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.741223 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.741235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.742448 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.742452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.742630 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.742777 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.742835 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.742891 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":153,"thread_start_us":95,"threads_started":1}
I20260810 07:15:04.745657 15076 tablet_service.cc:1467] Processing CreateTablet for tablet c6c04669f27045eab4919e18a99e4d3a (DEFAULT_TABLE table=test-table [id=47fed6760d014a688acaf0efe8c38370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.746060 15307 tablet_service.cc:1467] Processing CreateTablet for tablet c6c04669f27045eab4919e18a99e4d3a (DEFAULT_TABLE table=test-table [id=47fed6760d014a688acaf0efe8c38370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.746073 14827 tablet_service.cc:1467] Processing CreateTablet for tablet c6c04669f27045eab4919e18a99e4d3a (DEFAULT_TABLE table=test-table [id=47fed6760d014a688acaf0efe8c38370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.746784 15076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6c04669f27045eab4919e18a99e4d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.746976 15307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6c04669f27045eab4919e18a99e4d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.747041 14827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6c04669f27045eab4919e18a99e4d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.753854 15451 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Bootstrap starting.
I20260810 07:15:04.754671 15456 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Bootstrap starting.
I20260810 07:15:04.755626 15455 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Bootstrap starting.
I20260810 07:15:04.755985 15451 tablet_bootstrap.cc:654] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.756673 15456 tablet_bootstrap.cc:654] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.756927 15451 log.cc:824] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:04.757462 15456 log.cc:824] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:04.757603 15455 tablet_bootstrap.cc:654] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.758365 15455 log.cc:824] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:04.758730 15451 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: No bootstrap required, opened a new log
I20260810 07:15:04.758925 15451 ts_tablet_manager.cc:1397] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:04.759433 15456 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: No bootstrap required, opened a new log
I20260810 07:15:04.759616 15456 ts_tablet_manager.cc:1397] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 07:15:04.760365 15455 tablet_bootstrap.cc:492] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: No bootstrap required, opened a new log
I20260810 07:15:04.760552 15455 ts_tablet_manager.cc:1397] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:15:04.762028 15451 raft_consensus.cc:348] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.762292 15451 raft_consensus.cc:374] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.762347 15451 raft_consensus.cc:729] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d59021628d7432984a1e69f57037163, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.762766 15451 consensus_queue.cc:260] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.763448 15456 raft_consensus.cc:348] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.763643 15416 heartbeater.cc:503] Master 127.12.117.190:40575 was elected leader, sending a full tablet report...
I20260810 07:15:04.763702 15456 raft_consensus.cc:374] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.763759 15456 raft_consensus.cc:729] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf273a273de4e919f406c4e6b92388a, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.764046 15451 ts_tablet_manager.cc:1428] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:15:04.764211 15456 consensus_queue.cc:260] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.764204 15455 raft_consensus.cc:348] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.764431 15455 raft_consensus.cc:374] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.764484 15455 raft_consensus.cc:729] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7261b1caf4d14cdca5df9b8cc0e57c15, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.764858 15182 heartbeater.cc:503] Master 127.12.117.190:40575 was elected leader, sending a full tablet report...
I20260810 07:15:04.765120 15455 consensus_queue.cc:260] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.765229 15456 ts_tablet_manager.cc:1428] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:15:04.766076 15455 ts_tablet_manager.cc:1428] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260810 07:15:04.766390 14950 heartbeater.cc:503] Master 127.12.117.190:40575 was elected leader, sending a full tablet report...
I20260810 07:15:04.781898 15466 raft_consensus.cc:482] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:04.782065 15466 raft_consensus.cc:504] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
W20260810 07:15:04.782956 14952 tablet.cc:2369] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:04.783017 15466 leader_election.cc:288] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:04.787061 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.783250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41489 (local address 127.12.117.129:43231)
0810 07:15:04.783454 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.783470 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.783550 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.783764 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.783767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.783784 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:04.783886 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.783891 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.784747 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.784751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.785934 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.785945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.786654 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.786657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.786758 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.786784 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.786861 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.786934 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260810 07:15:04.787061 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.783402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:43231 (local address 127.12.117.131:41489)
0810 07:15:04.783499 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.783503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.783516 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:04.783648 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.783652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.783687 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.783745 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:04.784016 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.784654 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.786091 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.786545 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.786665 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.786715 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.786880 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.786946 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:04.787732 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.783430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:37627 (local address 127.12.117.130:39889)
0810 07:15:04.783536 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.783540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.783893 (+ 353us) server_negotiation.cc:407] Connection header received
0810 07:15:04.784003 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.784006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.784043 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.784114 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:04.784366 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.784844 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.786468 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.786996 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.787123 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.787189 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.787549 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.787626 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:04.787735 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.783250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:39889 (local address 127.12.117.129:37627)
0810 07:15:04.783804 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:04.783815 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:04.783913 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:04.784138 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:04.784141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:04.784160 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:04.784268 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.784273 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.784948 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.784951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.786362 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:04.786371 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.787322 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.787325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:04.787426 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.787448 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:15:04.787529 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:15:04.787596 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":241,"thread_start_us":120,"threads_started":1}
I20260810 07:15:04.787689 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6c04669f27045eab4919e18a99e4d3a" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163" is_pre_election: true
I20260810 07:15:04.787952 15343 raft_consensus.cc:2454] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 0.
I20260810 07:15:04.788275 15114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6c04669f27045eab4919e18a99e4d3a" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf273a273de4e919f406c4e6b92388a" is_pre_election: true
I20260810 07:15:04.788414 14741 leader_election.cc:302] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, 7261b1caf4d14cdca5df9b8cc0e57c15; no voters:
I20260810 07:15:04.788507 15114 raft_consensus.cc:2454] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 0.
I20260810 07:15:04.788671 15466 raft_consensus.cc:2781] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:04.788751 15466 raft_consensus.cc:482] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:04.788802 15466 raft_consensus.cc:3037] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:04.789824 15466 raft_consensus.cc:504] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.790151 15466 leader_election.cc:288] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 election: Requested vote from peers cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:04.790429 15114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6c04669f27045eab4919e18a99e4d3a" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf273a273de4e919f406c4e6b92388a"
I20260810 07:15:04.790457 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6c04669f27045eab4919e18a99e4d3a" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163"
I20260810 07:15:04.790549 15343 raft_consensus.cc:3037] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:04.790568 15114 raft_consensus.cc:3037] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:04.791591 15343 raft_consensus.cc:2454] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 1.
I20260810 07:15:04.791692 15114 raft_consensus.cc:2454] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 1.
I20260810 07:15:04.791858 14741 leader_election.cc:302] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, 7261b1caf4d14cdca5df9b8cc0e57c15; no voters:
I20260810 07:15:04.792024 15466 raft_consensus.cc:2781] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:04.792181 15466 raft_consensus.cc:686] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 LEADER]: Becoming Leader. State: Replica: 7261b1caf4d14cdca5df9b8cc0e57c15, State: Running, Role: LEADER
I20260810 07:15:04.792447 15466 consensus_queue.cc:237] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.795169 14590 catalog_manager.cc:5495] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:04.812516 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:04.809077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52816 (local address 127.12.117.129:37671)
0810 07:15:04.809184 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:04.809188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:04.809254 (+ 66us) server_negotiation.cc:407] Connection header received
0810 07:15:04.809459 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:04.809462 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:04.809509 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:04.809596 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:04.809832 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.810297 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.811143 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:04.811396 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:04.811486 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:04.811798 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:04.812148 (+ 350us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:04.812244 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:04.812351 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 07:15:04.812402 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:04.817680 15343 raft_consensus.cc:1261] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Refusing update from remote peer 7261b1caf4d14cdca5df9b8cc0e57c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:04.818224 15114 raft_consensus.cc:1261] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Refusing update from remote peer 7261b1caf4d14cdca5df9b8cc0e57c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:04.818295 15466 consensus_queue.cc:1035] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:04.818751 15466 consensus_queue.cc:1035] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:04.825182 15475 mvcc.cc:205] Tried to move back new op lower bound from 7316873645327130624 to 7316873645232136192. Current Snapshot: MvccSnapshot[applied={T|T < 7316873645327130624}]
I20260810 07:15:04.837726 14590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:04.837858 14590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:04.840077 14590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Sending DeleteTablet for 3 replicas of tablet c6c04669f27045eab4919e18a99e4d3a
I20260810 07:15:04.840782 14827 tablet_service.cc:1514] Processing DeleteTablet for tablet c6c04669f27045eab4919e18a99e4d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:04 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:52802
I20260810 07:15:04.840845 15076 tablet_service.cc:1514] Processing DeleteTablet for tablet c6c04669f27045eab4919e18a99e4d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:04 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55982
I20260810 07:15:04.840936 15307 tablet_service.cc:1514] Processing DeleteTablet for tablet c6c04669f27045eab4919e18a99e4d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:04 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60880
I20260810 07:15:04.841336 15489 tablet_replica.cc:330] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: stopping tablet replica
I20260810 07:15:04.841548 15486 tablet_replica.cc:330] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: stopping tablet replica
I20260810 07:15:04.841621 15489 raft_consensus.cc:2229] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:04.841820 15486 raft_consensus.cc:2229] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:04.842170 15486 raft_consensus.cc:2258] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:04.842254 14590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:04.842502 15488 tablet_replica.cc:330] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: stopping tablet replica
W20260810 07:15:04.842650 14590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:04.842765 15488 raft_consensus.cc:2229] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:04.843019 15488 raft_consensus.cc:2258] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:04.843637 15486 ts_tablet_manager.cc:1905] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:04.842355 15489 raft_consensus.cc:2258] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:04.845163 15488 ts_tablet_manager.cc:1905] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:04.847137 14827 tablet_service.cc:1467] Processing CreateTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 (DEFAULT_TABLE table=test-table [id=4afa81a706f84c478b181334d0cf789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.847429 14827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f2408e8f141f8821e8cfcf37979d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.847574 15076 tablet_service.cc:1467] Processing CreateTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 (DEFAULT_TABLE table=test-table [id=4afa81a706f84c478b181334d0cf789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.847808 15076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f2408e8f141f8821e8cfcf37979d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.848253 15488 ts_tablet_manager.cc:1918] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:04.848392 15488 log.cc:1196] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/c6c04669f27045eab4919e18a99e4d3a
I20260810 07:15:04.848764 15488 ts_tablet_manager.cc:1939] T c6c04669f27045eab4919e18a99e4d3a P cbf273a273de4e919f406c4e6b92388a: Deleting consensus metadata
I20260810 07:15:04.849726 15455 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Bootstrap starting.
I20260810 07:15:04.849967 15307 tablet_service.cc:1467] Processing CreateTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 (DEFAULT_TABLE table=test-table [id=4afa81a706f84c478b181334d0cf789a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:04.850203 15456 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Bootstrap starting.
I20260810 07:15:04.850251 15307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197f2408e8f141f8821e8cfcf37979d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:04.851092 15456 tablet_bootstrap.cc:654] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.851384 15455 tablet_bootstrap.cc:654] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.853544 15456 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: No bootstrap required, opened a new log
I20260810 07:15:04.853631 15456 ts_tablet_manager.cc:1397] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260810 07:15:04.853911 15486 ts_tablet_manager.cc:1918] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:04.854130 15486 log.cc:1196] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/c6c04669f27045eab4919e18a99e4d3a
I20260810 07:15:04.854084 15456 raft_consensus.cc:348] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.854189 15456 raft_consensus.cc:374] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.854230 15456 raft_consensus.cc:729] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf273a273de4e919f406c4e6b92388a, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.854367 15456 consensus_queue.cc:260] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.854619 15456 ts_tablet_manager.cc:1428] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:04.854628 15486 ts_tablet_manager.cc:1939] T c6c04669f27045eab4919e18a99e4d3a P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting consensus metadata
I20260810 07:15:04.856086 14568 catalog_manager.cc:4841] TS 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129:37671): tablet c6c04669f27045eab4919e18a99e4d3a (table test-table [id=47fed6760d014a688acaf0efe8c38370]) successfully deleted
I20260810 07:15:04.856602 15489 ts_tablet_manager.cc:1905] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:04.859107 15455 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: No bootstrap required, opened a new log
I20260810 07:15:04.859201 15455 ts_tablet_manager.cc:1397] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260810 07:15:04.859390 15489 ts_tablet_manager.cc:1918] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:04.859535 15489 log.cc:1196] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/c6c04669f27045eab4919e18a99e4d3a
I20260810 07:15:04.859670 15455 raft_consensus.cc:348] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.859779 15455 raft_consensus.cc:374] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.859820 15455 raft_consensus.cc:729] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7261b1caf4d14cdca5df9b8cc0e57c15, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.859978 15489 ts_tablet_manager.cc:1939] T c6c04669f27045eab4919e18a99e4d3a P 1d59021628d7432984a1e69f57037163: Deleting consensus metadata
I20260810 07:15:04.859941 15455 consensus_queue.cc:260] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.860186 15455 ts_tablet_manager.cc:1428] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:04.861654 14567 catalog_manager.cc:4841] TS cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889): tablet c6c04669f27045eab4919e18a99e4d3a (table test-table [id=47fed6760d014a688acaf0efe8c38370]) successfully deleted
I20260810 07:15:04.861936 14566 catalog_manager.cc:4841] TS 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489): tablet c6c04669f27045eab4919e18a99e4d3a (table test-table [id=47fed6760d014a688acaf0efe8c38370]) successfully deleted
I20260810 07:15:04.862675 15451 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Bootstrap starting.
I20260810 07:15:04.863559 15451 tablet_bootstrap.cc:654] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:04.864737 15451 tablet_bootstrap.cc:492] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: No bootstrap required, opened a new log
I20260810 07:15:04.864813 15451 ts_tablet_manager.cc:1397] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:04.865278 15451 raft_consensus.cc:348] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.865383 15451 raft_consensus.cc:374] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:04.865413 15451 raft_consensus.cc:729] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d59021628d7432984a1e69f57037163, State: Initialized, Role: FOLLOWER
I20260810 07:15:04.865521 15451 consensus_queue.cc:260] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:04.865761 15451 ts_tablet_manager.cc:1428] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260810 07:15:04.936170 15417 tablet.cc:2369] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:04.986596 15184 tablet.cc:2369] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:05.057334 15465 raft_consensus.cc:482] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:05.057478 15465 raft_consensus.cc:504] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.058498 15465 leader_election.cc:288] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129:37671), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:05.062088 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.058782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:59127 (local address 127.12.117.129:37671)
0810 07:15:05.058969 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.058974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.059034 (+ 60us) server_negotiation.cc:407] Connection header received
0810 07:15:05.059174 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.059177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.059220 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.059316 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.059564 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.060056 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061156 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.061549 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061652 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.061708 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.061890 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.061967 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 07:15:05.062124 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.058666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37671 (local address 127.12.117.130:59127)
0810 07:15:05.058948 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.058963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.059075 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.059326 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.059329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.059344 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.059462 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.059468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.060156 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.060159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.061037 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.061046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.061653 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.061768 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.061788 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.061850 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.061933 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":38,"thread_start_us":78,"threads_started":1}
I20260810 07:15:05.062139 15448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.058828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:56001 (local address 127.12.117.131:41489)
0810 07:15:05.058952 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.058957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.059137 (+ 180us) server_negotiation.cc:407] Connection header received
0810 07:15:05.059256 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.059284 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.059331 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.059389 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:05.059642 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.060126 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061198 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.061583 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061687 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.061739 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.061956 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.062019 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:05.062157 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.058722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41489 (local address 127.12.117.130:56001)
0810 07:15:05.059046 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.059066 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.059160 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.059421 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.059424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.059440 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:05.059538 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.059543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.060228 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.060232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.061069 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.061078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.061727 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.061731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.061827 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.061846 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.061924 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.061994 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":20,"thread_start_us":67,"threads_started":1}
I20260810 07:15:05.062541 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f2408e8f141f8821e8cfcf37979d0" candidate_uuid: "cbf273a273de4e919f406c4e6b92388a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163" is_pre_election: true
I20260810 07:15:05.062657 15343 raft_consensus.cc:2454] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf273a273de4e919f406c4e6b92388a in term 0.
I20260810 07:15:05.062918 14861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f2408e8f141f8821e8cfcf37979d0" candidate_uuid: "cbf273a273de4e919f406c4e6b92388a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" is_pre_election: true
I20260810 07:15:05.062978 14989 leader_election.cc:302] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, cbf273a273de4e919f406c4e6b92388a; no voters:
I20260810 07:15:05.063114 14861 raft_consensus.cc:2454] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf273a273de4e919f406c4e6b92388a in term 0.
I20260810 07:15:05.063252 15465 raft_consensus.cc:2781] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:05.063339 15465 raft_consensus.cc:482] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:05.063377 15465 raft_consensus.cc:3037] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.064303 15465 raft_consensus.cc:504] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.064613 15465 leader_election.cc:288] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [CANDIDATE]: Term 1 election: Requested vote from peers 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129:37671), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:05.064960 14861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f2408e8f141f8821e8cfcf37979d0" candidate_uuid: "cbf273a273de4e919f406c4e6b92388a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15"
I20260810 07:15:05.064941 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197f2408e8f141f8821e8cfcf37979d0" candidate_uuid: "cbf273a273de4e919f406c4e6b92388a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163"
I20260810 07:15:05.065071 15343 raft_consensus.cc:3037] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.065080 14861 raft_consensus.cc:3037] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.065929 15343 raft_consensus.cc:2454] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf273a273de4e919f406c4e6b92388a in term 1.
I20260810 07:15:05.066092 14861 raft_consensus.cc:2454] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf273a273de4e919f406c4e6b92388a in term 1.
I20260810 07:15:05.066174 14989 leader_election.cc:302] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, cbf273a273de4e919f406c4e6b92388a; no voters:
I20260810 07:15:05.066342 15465 raft_consensus.cc:2781] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:05.066512 15465 raft_consensus.cc:686] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 1 LEADER]: Becoming Leader. State: Replica: cbf273a273de4e919f406c4e6b92388a, State: Running, Role: LEADER
I20260810 07:15:05.066745 15465 consensus_queue.cc:237] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.068768 14590 catalog_manager.cc:5495] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a reported cstate change: term changed from 0 to 1, leader changed from <none> to cbf273a273de4e919f406c4e6b92388a (127.12.117.130). New cstate: current_term: 1 leader_uuid: "cbf273a273de4e919f406c4e6b92388a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:15:05 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346103, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:05.108408 15508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.100656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:40575 (local address 127.0.0.1:55866)
0810 07:15:05.100913 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.100924 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.101028 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.101242 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.101244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.101662 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:05.101768 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.101774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.102493 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.102496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.103028 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.103035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.103482 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.103485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.103614 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.104119 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:05.104130 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:05.105470 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:05.106972 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.106982 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.106988 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.107203 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.107466 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:05.107469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:05.107471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:05.107545 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:05.107875 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:05.107881 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:05.107987 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.108081 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.108224 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":17,"thread_start_us":95,"threads_started":1}
I20260810 07:15:05.108407 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.100734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.12.117.190:40575)
0810 07:15:05.100854 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.100858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.101013 (+ 155us) server_negotiation.cc:407] Connection header received
0810 07:15:05.101111 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.101114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.101152 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.101214 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.101890 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.102390 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.103133 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.103384 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.103470 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.105588 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.105591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.105593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.105617 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.106876 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.107310 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.107314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.107315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.107383 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.107620 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.107623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.107625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.107782 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.107865 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.108140 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.108274 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":44}
I20260810 07:15:05.111946 14590 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55866:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:15:05.112641 14590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:05.116902 14827 tablet_service.cc:1467] Processing CreateTablet for tablet 26262292dd8b4fe29e6059ce85fa5caa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54e5a488af4745bf91dc9be16eb4181e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:05.116977 15076 tablet_service.cc:1467] Processing CreateTablet for tablet 26262292dd8b4fe29e6059ce85fa5caa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54e5a488af4745bf91dc9be16eb4181e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:05.117096 15307 tablet_service.cc:1467] Processing CreateTablet for tablet 26262292dd8b4fe29e6059ce85fa5caa (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=54e5a488af4745bf91dc9be16eb4181e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:05.117259 14827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26262292dd8b4fe29e6059ce85fa5caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.117348 15076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26262292dd8b4fe29e6059ce85fa5caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.117409 15307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26262292dd8b4fe29e6059ce85fa5caa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:05.120110 15456 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: Bootstrap starting.
I20260810 07:15:05.120195 15455 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: Bootstrap starting.
I20260810 07:15:05.120972 15455 tablet_bootstrap.cc:654] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.120997 15456 tablet_bootstrap.cc:654] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.121539 15451 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: Bootstrap starting.
I20260810 07:15:05.122021 15455 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: No bootstrap required, opened a new log
I20260810 07:15:05.122118 15455 ts_tablet_manager.cc:1397] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:15:05.122414 15451 tablet_bootstrap.cc:654] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:05.122498 15455 raft_consensus.cc:348] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.122619 15455 raft_consensus.cc:374] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.122653 15455 raft_consensus.cc:729] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7261b1caf4d14cdca5df9b8cc0e57c15, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.122776 15455 consensus_queue.cc:260] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.123015 15455 ts_tablet_manager.cc:1428] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:05.122982 15466 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.123121 15466 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.123324 15456 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: No bootstrap required, opened a new log
I20260810 07:15:05.123415 15456 ts_tablet_manager.cc:1397] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 07:15:05.123760 15451 tablet_bootstrap.cc:492] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: No bootstrap required, opened a new log
I20260810 07:15:05.123850 15451 ts_tablet_manager.cc:1397] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:05.123836 15456 raft_consensus.cc:348] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.123946 15456 raft_consensus.cc:374] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.123986 15456 raft_consensus.cc:729] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf273a273de4e919f406c4e6b92388a, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.124106 15456 consensus_queue.cc:260] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.124334 15456 ts_tablet_manager.cc:1428] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:05.124285 15451 raft_consensus.cc:348] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.124395 15451 raft_consensus.cc:374] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:05.124439 15451 raft_consensus.cc:729] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d59021628d7432984a1e69f57037163, State: Initialized, Role: FOLLOWER
I20260810 07:15:05.124522 15465 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.124570 15451 consensus_queue.cc:260] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.124644 15465 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.124825 15451 ts_tablet_manager.cc:1428] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:05.125412 15463 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.125574 15463 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.158206 15466 raft_consensus.cc:482] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:05.158309 15466 raft_consensus.cc:504] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.158622 15466 leader_election.cc:288] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:05.159010 15114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26262292dd8b4fe29e6059ce85fa5caa" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf273a273de4e919f406c4e6b92388a" is_pre_election: true
I20260810 07:15:05.159015 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26262292dd8b4fe29e6059ce85fa5caa" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163" is_pre_election: true
I20260810 07:15:05.159160 15114 raft_consensus.cc:2454] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 0.
I20260810 07:15:05.159161 15343 raft_consensus.cc:2454] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 0.
I20260810 07:15:05.159446 14741 leader_election.cc:302] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, 7261b1caf4d14cdca5df9b8cc0e57c15; no voters:
I20260810 07:15:05.159627 15466 raft_consensus.cc:2781] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:05.159684 15466 raft_consensus.cc:482] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:05.159729 15466 raft_consensus.cc:3037] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.160665 15466 raft_consensus.cc:504] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.160964 15466 leader_election.cc:288] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 election: Requested vote from peers cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889), 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489)
I20260810 07:15:05.161319 15343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26262292dd8b4fe29e6059ce85fa5caa" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d59021628d7432984a1e69f57037163"
I20260810 07:15:05.161319 15114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26262292dd8b4fe29e6059ce85fa5caa" candidate_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf273a273de4e919f406c4e6b92388a"
I20260810 07:15:05.161437 15343 raft_consensus.cc:3037] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.161437 15114 raft_consensus.cc:3037] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:05.162510 15114 raft_consensus.cc:2454] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 1.
I20260810 07:15:05.162510 15343 raft_consensus.cc:2454] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7261b1caf4d14cdca5df9b8cc0e57c15 in term 1.
I20260810 07:15:05.162772 14741 leader_election.cc:302] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d59021628d7432984a1e69f57037163, 7261b1caf4d14cdca5df9b8cc0e57c15; no voters:
I20260810 07:15:05.162945 15466 raft_consensus.cc:2781] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:05.163049 15466 raft_consensus.cc:686] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 LEADER]: Becoming Leader. State: Replica: 7261b1caf4d14cdca5df9b8cc0e57c15, State: Running, Role: LEADER
I20260810 07:15:05.163182 15466 consensus_queue.cc:237] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } }
I20260810 07:15:05.163887 15470 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: TxnStatusTablet state changed. Reason: New leader 7261b1caf4d14cdca5df9b8cc0e57c15. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.163981 15470 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:05.164235 15522 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:15:05.164419 15522 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:15:05.164837 14596 catalog_manager.cc:5495] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:05.175638 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.172266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.12.117.129:37671)
0810 07:15:05.172377 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.172381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.172393 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:05.172559 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.172562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.172601 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.172660 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:05.172904 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.173389 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.174617 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.174833 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.174906 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.175210 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.175305 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.175360 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.175490 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.175539 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:15:05.175684 15508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.172160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37671 (local address 127.0.0.1:52826)
0810 07:15:05.172321 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.172336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.172451 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.172694 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.172697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.172707 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:05.172816 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.172822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.173485 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.173489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.174494 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.174506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.174923 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.174926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.175018 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.175073 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.175376 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.175389 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.175464 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.175548 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:15:05.177882 15343 raft_consensus.cc:1261] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Refusing update from remote peer 7261b1caf4d14cdca5df9b8cc0e57c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.178026 15114 raft_consensus.cc:1261] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Refusing update from remote peer 7261b1caf4d14cdca5df9b8cc0e57c15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.178297 15470 consensus_queue.cc:1035] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.178422 15466 consensus_queue.cc:1035] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.179718 15465 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: TxnStatusTablet state changed. Reason: New leader 7261b1caf4d14cdca5df9b8cc0e57c15. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.179822 15465 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.179787 15463 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: TxnStatusTablet state changed. Reason: New leader 7261b1caf4d14cdca5df9b8cc0e57c15. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.179888 15463 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.181455 15470 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.181553 15470 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:05.183368 15465 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.183463 15465 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P cbf273a273de4e919f406c4e6b92388a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.185452 15470 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.185552 15470 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 7261b1caf4d14cdca5df9b8cc0e57c15: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:05.186767 15463 tablet_replica.cc:439] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 } } peers { permanent_uuid: "cbf273a273de4e919f406c4e6b92388a" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39889 } } peers { permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 } } }
I20260810 07:15:05.186867 15463 tablet_replica.cc:442] T 26262292dd8b4fe29e6059ce85fa5caa P 1d59021628d7432984a1e69f57037163: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:05.190912 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.187225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.12.117.130:39889)
0810 07:15:05.187346 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.187351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.187364 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:05.187431 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.187434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.187497 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.187602 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:05.187893 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.188512 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.189394 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.189715 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.189802 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.190073 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.190548 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.190661 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.190749 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.190799 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:15:05.200822 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.197278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:35177 (local address 127.12.117.129:37671)
0810 07:15:05.197388 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.197393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.197643 (+ 250us) server_negotiation.cc:407] Connection header received
0810 07:15:05.197768 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.197771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.197820 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.197895 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:05.198117 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.198578 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.199817 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.200088 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.200169 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.200483 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.200544 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.200597 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.200692 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.200733 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:05.200915 15537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.197173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37671 (local address 127.12.117.130:35177)
0810 07:15:05.197558 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.197572 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.197670 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.197909 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.197911 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.197919 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:05.198030 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.198036 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.198663 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.198666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.199683 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.199695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.200210 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.200215 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.200311 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.200363 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.200616 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.200622 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.200675 (+ 53us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.200753 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":113,"threads_started":1}
I20260810 07:15:05.207508 15343 raft_consensus.cc:1261] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Refusing update from remote peer cbf273a273de4e919f406c4e6b92388a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.207610 14861 raft_consensus.cc:1261] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Refusing update from remote peer cbf273a273de4e919f406c4e6b92388a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:05.207928 15465 consensus_queue.cc:1035] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d59021628d7432984a1e69f57037163" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.208106 15500 consensus_queue.cc:1035] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7261b1caf4d14cdca5df9b8cc0e57c15" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:05.213649 15477 mvcc.cc:205] Tried to move back new op lower bound from 7316873646925447168 to 7316873646355681280. Current Snapshot: MvccSnapshot[applied={T|T < 7316873646925447168}]
I20260810 07:15:05.229890 15443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.226402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59195 (local address 127.12.117.130:39889)
0810 07:15:05.226512 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.226517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.226766 (+ 249us) server_negotiation.cc:407] Connection header received
0810 07:15:05.226873 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.226876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.226917 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.226975 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:05.227200 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.227822 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.228906 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.229142 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.229228 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.229506 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.229576 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.229633 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.229721 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.229770 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:05.229920 15549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.226347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:39889 (local address 127.12.117.129:59195)
0810 07:15:05.226683 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:05.226695 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:05.226795 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:05.226992 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:05.226995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:05.227007 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:05.227103 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.227109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.227926 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.227929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.228807 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:05.228818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.229239 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.229243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:05.229339 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.229384 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:05.229647 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:05.229655 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:15:05.229713 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 07:15:05.229794 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":132,"threads_started":1}
I20260810 07:15:05.257262 14596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:05.257402 14596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:05.259634 14596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2fed4831a5d44d118cce0102787df6d8: Sending DeleteTablet for 3 replicas of tablet 197f2408e8f141f8821e8cfcf37979d0
I20260810 07:15:05.260203 14827 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:52802
I20260810 07:15:05.260368 15486 tablet_replica.cc:330] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: stopping tablet replica
I20260810 07:15:05.260329 15307 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60880
I20260810 07:15:05.260481 15486 raft_consensus.cc:2229] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:05.260495 15489 tablet_replica.cc:330] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: stopping tablet replica
I20260810 07:15:05.260574 15486 raft_consensus.cc:2258] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:05.260604 15489 raft_consensus.cc:2229] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:05.260706 15489 raft_consensus.cc:2258] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:05.261278 15486 ts_tablet_manager.cc:1905] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:05.261358 15489 ts_tablet_manager.cc:1905] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:05.263845 15076 tablet_service.cc:1514] Processing DeleteTablet for tablet 197f2408e8f141f8821e8cfcf37979d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:05 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55982
I20260810 07:15:05.264010 15488 tablet_replica.cc:330] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: stopping tablet replica
I20260810 07:15:05.264039 15486 ts_tablet_manager.cc:1918] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:05.264039 15489 ts_tablet_manager.cc:1918] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:05.264108 15489 log.cc:1196] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/197f2408e8f141f8821e8cfcf37979d0
I20260810 07:15:05.264104 15488 raft_consensus.cc:2229] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:05.264346 15489 ts_tablet_manager.cc:1939] T 197f2408e8f141f8821e8cfcf37979d0 P 1d59021628d7432984a1e69f57037163: Deleting consensus metadata
I20260810 07:15:05.264401 15488 raft_consensus.cc:2258] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:05.264993 14566 catalog_manager.cc:4841] TS 1d59021628d7432984a1e69f57037163 (127.12.117.131:41489): tablet 197f2408e8f141f8821e8cfcf37979d0 (table test-table [id=4afa81a706f84c478b181334d0cf789a]) successfully deleted
I20260810 07:15:05.265407 15486 log.cc:1196] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/197f2408e8f141f8821e8cfcf37979d0
I20260810 07:15:05.265691 15486 ts_tablet_manager.cc:1939] T 197f2408e8f141f8821e8cfcf37979d0 P 7261b1caf4d14cdca5df9b8cc0e57c15: Deleting consensus metadata
I20260810 07:15:05.265856 15488 ts_tablet_manager.cc:1905] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:05.266275 14568 catalog_manager.cc:4841] TS 7261b1caf4d14cdca5df9b8cc0e57c15 (127.12.117.129:37671): tablet 197f2408e8f141f8821e8cfcf37979d0 (table test-table [id=4afa81a706f84c478b181334d0cf789a]) successfully deleted
I20260810 07:15:05.268525 15488 ts_tablet_manager.cc:1918] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:05.268600 15488 log.cc:1196] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/197f2408e8f141f8821e8cfcf37979d0
I20260810 07:15:05.268841 15488 ts_tablet_manager.cc:1939] T 197f2408e8f141f8821e8cfcf37979d0 P cbf273a273de4e919f406c4e6b92388a: Deleting consensus metadata
I20260810 07:15:05.269621 14567 catalog_manager.cc:4841] TS cbf273a273de4e919f406c4e6b92388a (127.12.117.130:39889): tablet 197f2408e8f141f8821e8cfcf37979d0 (table test-table [id=4afa81a706f84c478b181334d0cf789a]) successfully deleted
Aug 10 07:15:05 dist-test-slave-gjrz krb5kdc[14520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346104, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:05.278688 15450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.266741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.12.117.129:37671)
0810 07:15:05.266840 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.266844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.267571 (+ 727us) server_negotiation.cc:407] Connection header received
0810 07:15:05.267734 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.267737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.267780 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.267824 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.268826 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.269565 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.270773 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.271118 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.271224 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.274865 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.274876 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.274892 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.274919 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.276893 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.277356 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.277362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.277364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.277457 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.277703 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.277706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.277708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.277997 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.278203 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.278397 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.278523 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 07:15:05.279078 14943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52840
I20260810 07:15:05.304684 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:05.281019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55868 (local address 127.12.117.190:40575)
0810 07:15:05.281131 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:05.281135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:05.295599 (+ 14464us) server_negotiation.cc:407] Connection header received
0810 07:15:05.295811 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:05.295814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:05.295867 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:05.295924 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:05.296921 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.297731 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.298880 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:05.299229 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:05.299358 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:05.301139 (+ 1781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:05.301143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:05.301146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:05.301173 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:05.302821 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.303305 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.303308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.303310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.303390 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:05.303672 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:05.303675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:05.303677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:05.303871 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:05.303960 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:05.304421 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 07:15:05.304534 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 07:15:05.305533 14596 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55868
I20260810 07:15:05.307020 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14721
I20260810 07:15:05.322228 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14968
I20260810 07:15:05.335953 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15202
I20260810 07:15:05.356237 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14536
2026-08-10T07:15:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2819 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:05 dist-test-slave-gjrz krb5kdc[15561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:05 dist-test-slave-gjrz krb5kdc[15561](info): set up 2 sockets
Aug 10 07:15:05 dist-test-slave-gjrz krb5kdc[15561](info): commencing operation
krb5kdc: starting...
W20260810 07:15:06.514588 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:06 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:06.624061 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34049
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:34049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:06.728853 15584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.729077 15584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.729131 15584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.732995 15584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:06.733039 15584 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:15:06.733063 15584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.733086 15584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.733119 15584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:06.733145 15584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:06.738096 15584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:34049
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:06.738931 15584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.739809 15584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:06.747241 15593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:06.747314 15599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.747414 15584 server_base.cc:1034] running on GCE node
W20260810 07:15:06.747315 15597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:06.747995 15584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:06.748960 15584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:06.750102 15584 hybrid_clock.cc:648] HybridClock initialized: now 1786346106750082 us; error 34 us; skew 500 ppm
Aug 10 07:15:06 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346106, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:06.752648 15584 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:06.753371 15584 webserver.cc:460] Webserver started at http://127.12.117.190:46287/ using document root <none> and password file <none>
I20260810 07:15:06.753829 15584 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:06.753883 15584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:06.754032 15584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:06.755857 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "7d81f75aafbc489da85b04bb5ae2d7d6"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-gjrz"
server_key: "68c51d87d91f19c144a23441e61a0038"
server_key_iv: "820ae20cd8b87558e8643d5e94251347"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.756328 15584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "7d81f75aafbc489da85b04bb5ae2d7d6"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-gjrz"
server_key: "68c51d87d91f19c144a23441e61a0038"
server_key_iv: "820ae20cd8b87558e8643d5e94251347"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.759649 15584 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:06.761744 15611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:06.762852 15584 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:06.762955 15584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "7d81f75aafbc489da85b04bb5ae2d7d6"
format_stamp: "Formatted at 2026-08-10 07:15:06 on dist-test-slave-gjrz"
server_key: "68c51d87d91f19c144a23441e61a0038"
server_key_iv: "820ae20cd8b87558e8643d5e94251347"
server_key_version: "encryptionkey@0"
I20260810 07:15:06.763054 15584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:06.786000 15584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:06.788559 15584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:06.788698 15584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:06.795680 15584 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:34049
I20260810 07:15:06.795727 15705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:34049 every 8 connection(s)
I20260810 07:15:06.796509 15584 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:06.798877 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15584
I20260810 07:15:06.798990 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:06.799181 15707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:06.799297 12758 external_mini_cluster.cc:1428] Setting key 42ef37adf33533eb6e881e6bcc302a12
I20260810 07:15:06.805161 15707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Bootstrap starting.
I20260810 07:15:06.807309 15707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Neither blocks nor log segments found. Creating new log.
Aug 10 07:15:06 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:06.807998 15707 log.cc:824] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:06.809832 15707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: No bootstrap required, opened a new log
I20260810 07:15:06.812718 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:06.800610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.12.117.190:34049)
0810 07:15:06.801016 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:06.801030 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:06.801065 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:06.801779 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:06.801797 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:06.802112 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:06.802404 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:06.803165 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:06.804172 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:06.805005 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:06.805490 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:06.805643 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:06.807857 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:06.807862 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:06.807875 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:06.807901 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:06.810167 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:06.810558 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:06.810563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:06.810567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:06.810682 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:06.810939 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:06.810942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:06.810943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:06.811351 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:06.811533 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:06.811809 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 07:15:06.812071 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":137,"threads_started":1}
I20260810 07:15:06.812623 15707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } }
I20260810 07:15:06.812804 15707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:06.812848 15707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d81f75aafbc489da85b04bb5ae2d7d6, State: Initialized, Role: FOLLOWER
I20260810 07:15:06.813305 15707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } }
I20260810 07:15:06.813458 15707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:06.813524 15707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:06.813616 15707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:06.814402 15707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } }
I20260810 07:15:06.814723 15707 leader_election.cc:302] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d81f75aafbc489da85b04bb5ae2d7d6; no voters:
I20260810 07:15:06.815032 15707 leader_election.cc:288] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:06.815135 15722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:06.815356 15722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [term 1 LEADER]: Becoming Leader. State: Replica: 7d81f75aafbc489da85b04bb5ae2d7d6, State: Running, Role: LEADER
I20260810 07:15:06.815632 15722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } }
I20260810 07:15:06.816015 15707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:06.817066 15725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d81f75aafbc489da85b04bb5ae2d7d6. Latest consensus state: current_term: 1 leader_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } } }
I20260810 07:15:06.817171 15725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:06.817458 15724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d81f75aafbc489da85b04bb5ae2d7d6" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34049 } } }
I20260810 07:15:06.817548 15724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:06.817924 15733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:06.820654 15733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:06.826090 15733 catalog_manager.cc:1347] Generated new cluster ID: ca08cd13d85546bd986e39f6da66d356
I20260810 07:15:06.826160 15733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:06.841830 15733 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:06.842669 15733 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:06.852738 15733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Generated new TSK 0
I20260810 07:15:06.853307 15733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:06.877674 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34049
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:06.983613 15752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:06.983824 15752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:06.983870 15752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:06.987692 15752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:06.987744 15752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:06.987785 15752 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:15:06.987812 15752 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:06.987860 15752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:06.992866 15752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:06.993726 15752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:06.994602 15752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.003291 15758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.003362 15752 server_base.cc:1034] running on GCE node
W20260810 07:15:07.003368 15757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.003583 15763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.003916 15752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.004477 15752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.005640 15752 hybrid_clock.cc:648] HybridClock initialized: now 1786346107005624 us; error 28 us; skew 500 ppm
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:07.008411 15752 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:07.009184 15752 webserver.cc:460] Webserver started at http://127.12.117.129:35299/ using document root <none> and password file <none>
I20260810 07:15:07.009706 15752 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.009770 15752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.009944 15752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.011821 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "bf61e049cfd6ed3b0807b621e17dd194"
server_key_iv: "62716679c411302226fa76bb74dee199"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.012339 15752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "bf61e049cfd6ed3b0807b621e17dd194"
server_key_iv: "62716679c411302226fa76bb74dee199"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.015811 15752 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:15:07.018059 15770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.019147 15752 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:07.019248 15752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "bf61e049cfd6ed3b0807b621e17dd194"
server_key_iv: "62716679c411302226fa76bb74dee199"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.019364 15752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.031253 15752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.033864 15752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.034021 15752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.035323 15752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:07.035372 15752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.035421 15752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:07.035449 15752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:07.045066 15752 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:42919
I20260810 07:15:07.045123 15947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:42919 every 8 connection(s)
I20260810 07:15:07.046051 15752 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:07.048430 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.036077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47269 (local address 127.12.117.190:34049)
0810 07:15:07.036228 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.036234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.037025 (+ 791us) server_negotiation.cc:407] Connection header received
0810 07:15:07.037971 (+ 946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.037975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.038035 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.038083 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.039708 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.040502 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.042155 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.042478 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.042577 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.045243 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.045247 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.045250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.045277 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.046710 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.047161 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.047164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.047166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.047247 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.047544 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.047546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.047547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.047723 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.047795 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.048198 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.048308 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:15:07.049118 15783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.036410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.129:47269)
0810 07:15:07.036861 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.036909 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.037729 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.038440 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.038450 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.038897 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.039541 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.039576 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.040655 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.040659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.042029 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.042041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.042623 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.042628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.042818 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.043367 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.043388 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.045080 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.046816 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.046821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.046831 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.047062 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.047340 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.047342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.047344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.047466 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.047798 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.047803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.048005 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.048156 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.048372 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260810 07:15:07.050374 15948 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34049
I20260810 07:15:07.050503 15948 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:07.050935 15948 heartbeater.cc:511] Master 127.12.117.190:34049 requested a full tablet report, sending...
I20260810 07:15:07.052488 15636 ts_manager.cc:194] Registered new tserver with Master: f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919)
I20260810 07:15:07.053131 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15752
I20260810 07:15:07.053236 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:07.053476 12758 external_mini_cluster.cc:1428] Setting key 954bca63e5fcc711222d9c0bcb57fbbe
I20260810 07:15:07.053835 15636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:47269
I20260810 07:15:07.060009 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.052371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:40121 (local address 127.12.117.190:34049)
0810 07:15:07.052486 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.052490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.052658 (+ 168us) server_negotiation.cc:407] Connection header received
0810 07:15:07.052749 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.052751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.052788 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.052854 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.054170 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.054820 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.055889 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.056101 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.056178 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.057221 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.057225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.057227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.057253 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.058613 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.059033 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.059035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.059037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.059101 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.059358 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.059360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.059361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.059505 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.059583 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.059825 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.059909 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:07.060092 15957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.052298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.129:40121)
0810 07:15:07.052565 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.052580 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.052674 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.052889 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.052893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.053960 (+ 1067us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.054069 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.054077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.054941 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.054944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.055767 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.055778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.056202 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.056206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.056289 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.056695 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.056708 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.057083 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.058718 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.058722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.058724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.058953 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.059171 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.059174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.059176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.059289 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.059594 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.059597 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.059687 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.059782 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.059903 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:07.079600 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34049
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:07.186455 15962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:07.186679 15962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:07.186740 15962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:07.190666 15962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:07.190716 15962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:07.190768 15962 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:15:07.190795 15962 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:07.190841 15962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:07.195797 15962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:07.196666 15962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:07.197570 15962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.206496 15970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.206529 15971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.206657 15962 server_base.cc:1034] running on GCE node
W20260810 07:15:07.206496 15973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.207031 15962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.207661 15962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.208817 15962 hybrid_clock.cc:648] HybridClock initialized: now 1786346107208801 us; error 27 us; skew 500 ppm
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:07.211356 15962 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:07.212124 15962 webserver.cc:460] Webserver started at http://127.12.117.130:44311/ using document root <none> and password file <none>
I20260810 07:15:07.212622 15962 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.212687 15962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.212863 15962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.214711 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "9f130ee7d2ae402f992256590408d94d"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "745832079712db0fe69ba47da0e9ca9b"
server_key_iv: "a8c7c3ba22282c4c0f4a08cbeea85580"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.215227 15962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "9f130ee7d2ae402f992256590408d94d"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "745832079712db0fe69ba47da0e9ca9b"
server_key_iv: "a8c7c3ba22282c4c0f4a08cbeea85580"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.218760 15962 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:07.220992 15984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.222123 15962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:07.222218 15962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "9f130ee7d2ae402f992256590408d94d"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "745832079712db0fe69ba47da0e9ca9b"
server_key_iv: "a8c7c3ba22282c4c0f4a08cbeea85580"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.222313 15962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.235762 15962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.238204 15962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.238355 15962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.239701 15962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:07.239749 15962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.239784 15962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:07.239810 15962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:07.249505 15962 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:43429
I20260810 07:15:07.249630 16162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:43429 every 8 connection(s)
I20260810 07:15:07.250320 15962 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:07.253159 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.240316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:59629 (local address 127.12.117.190:34049)
0810 07:15:07.240467 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.240473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.241672 (+ 1199us) server_negotiation.cc:407] Connection header received
0810 07:15:07.242843 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.242846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.242904 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.242993 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.244497 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.245304 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.247017 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.247244 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.247364 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.250057 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.250061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.250064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.250089 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.251513 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.252000 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.252003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.252004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.252094 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.252355 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.252357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.252358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.252508 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.252575 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.252941 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.253032 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:07.253918 16000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.241058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.130:59629)
0810 07:15:07.241502 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.241565 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.242598 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.243165 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.243174 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.243680 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.244320 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.244342 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.245456 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.245460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.246877 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.246889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.247409 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.247414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.247641 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.248224 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.248246 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.249875 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.251603 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.251608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.251620 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.251892 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.252173 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.252175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.252176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.252298 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.252572 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.252576 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.252768 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.252906 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.253151 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
I20260810 07:15:07.254801 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15962
I20260810 07:15:07.254912 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:07.255137 16163 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34049
I20260810 07:15:07.255249 12758 external_mini_cluster.cc:1428] Setting key 5e72182dbd38f125ccb18e578ac3e0b1
I20260810 07:15:07.255306 16163 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:07.255828 16163 heartbeater.cc:511] Master 127.12.117.190:34049 requested a full tablet report, sending...
I20260810 07:15:07.256860 15636 ts_manager.cc:194] Registered new tserver with Master: 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:07.257516 15636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:59629
I20260810 07:15:07.263975 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.257088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.130:47755)
0810 07:15:07.257432 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.257444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.257532 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.257731 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.257734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.257927 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.258023 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.258029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.259032 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.259035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.259790 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.259800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.260215 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.260218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.260308 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.260690 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.260700 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.260959 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.262654 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.262656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.262658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.262837 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.263101 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.263103 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.263104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.263166 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.263611 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.263613 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.263685 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.263763 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.263857 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260810 07:15:07.264092 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.257147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47755 (local address 127.12.117.190:34049)
0810 07:15:07.257252 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.257256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.257513 (+ 257us) server_negotiation.cc:407] Connection header received
0810 07:15:07.257601 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.257604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.257644 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.257706 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.258163 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.258875 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.259901 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.260120 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.260202 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.261083 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.261086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.261088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.261112 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.262552 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.262925 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.262928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.262930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.263023 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.263236 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.263239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.263241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.263529 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.263628 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.263829 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.263938 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:07.280324 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34049
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:07.386561 16182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:07.386770 16182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:07.386811 16182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:07.390729 16182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:07.390774 16182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:07.390811 16182 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 07:15:07.390829 16182 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:07.390858 16182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:07.395730 16182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:34049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:07.396538 16182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:07.397346 16182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:07.406090 16194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.406209 16182 server_base.cc:1034] running on GCE node
W20260810 07:15:07.406090 16192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:07.406220 16191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:07.406667 16182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:07.407231 16182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:07.408403 16182 hybrid_clock.cc:648] HybridClock initialized: now 1786346107408389 us; error 26 us; skew 500 ppm
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:07.410853 16182 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:07.411656 16182 webserver.cc:460] Webserver started at http://127.12.117.131:37163/ using document root <none> and password file <none>
I20260810 07:15:07.412137 16182 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:07.412202 16182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:07.412402 16182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:07.414245 16182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "b0df336b3d0145aaadf637bbbdcba656"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "b97b492790208730a3b1396d2d570589"
server_key_iv: "642b9c7a63ebc40cff79f7e17f8ec3c2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.414755 16182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "b0df336b3d0145aaadf637bbbdcba656"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "b97b492790208730a3b1396d2d570589"
server_key_iv: "642b9c7a63ebc40cff79f7e17f8ec3c2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.418347 16182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:07.420565 16207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.421655 16182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:07.421769 16182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "b0df336b3d0145aaadf637bbbdcba656"
format_stamp: "Formatted at 2026-08-10 07:15:07 on dist-test-slave-gjrz"
server_key: "b97b492790208730a3b1396d2d570589"
server_key_iv: "642b9c7a63ebc40cff79f7e17f8ec3c2"
server_key_version: "encryptionkey@0"
I20260810 07:15:07.421861 16182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:07.456420 16182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:07.459283 16182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:07.459448 16182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:07.460810 16182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:07.460860 16182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:07.460911 16182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:07.460942 16182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:07.471167 16182 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:40889
I20260810 07:15:07.471206 16384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:40889 every 8 connection(s)
I20260810 07:15:07.471992 16182 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:07.474241 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.461525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:40265 (local address 127.12.117.190:34049)
0810 07:15:07.461635 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.461638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.462650 (+ 1012us) server_negotiation.cc:407] Connection header received
0810 07:15:07.463604 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.463608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.463667 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.463751 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.465275 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.466139 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.467904 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.468118 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.468210 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.470789 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.470794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.470797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.470828 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.472276 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.472823 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.472826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.472827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.472910 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.473222 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.473225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.473227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.473407 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.473494 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.473965 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.474084 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:07.474916 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.462039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.131:40265)
0810 07:15:07.462507 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.462548 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.463408 (+ 860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.463964 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.463974 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.464458 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.465116 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.465138 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.466279 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.466283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.467785 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.467798 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.468260 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.468265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.468447 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.468944 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.468965 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.470615 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.472383 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.472389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.472402 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.472720 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.472995 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.472999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.473001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.473133 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.473507 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.473514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.473732 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.473893 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.474197 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":119,"threads_started":1}
I20260810 07:15:07.476012 16387 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34049
I20260810 07:15:07.476153 16387 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:07.476202 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16182
I20260810 07:15:07.476320 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:07.476630 12758 external_mini_cluster.cc:1428] Setting key 9351630dba0aad1a899b1347077d2fa3
I20260810 07:15:07.476639 16387 heartbeater.cc:511] Master 127.12.117.190:34049 requested a full tablet report, sending...
I20260810 07:15:07.477694 15637 ts_manager.cc:194] Registered new tserver with Master: b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:07.478432 15637 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:40265
I20260810 07:15:07.478811 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:07.486713 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.478638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:60977 (local address 127.12.117.190:34049)
0810 07:15:07.478742 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.478746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.478758 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:07.478788 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.478791 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.478830 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.478911 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.480905 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.481477 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.482550 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.482754 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.482864 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.483820 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.483824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.483826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.483849 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.485287 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.485679 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.485682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.485684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.485769 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.485986 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.485989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.485991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.486191 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.486270 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.486528 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.486611 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:15:07.486876 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.478174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.12.117.131:60977)
0810 07:15:07.478427 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.478439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.478510 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.478953 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.478956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.480642 (+ 1686us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.480757 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.480765 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.481602 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.481606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.482424 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.482436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.482876 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.482879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.482973 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.483403 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.483415 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.483695 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.485410 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.485412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.485414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.485593 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.485847 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.485849 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.485851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.485913 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.486281 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.486284 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.486380 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.486482 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.486613 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:07.506788 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.499531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.12.117.190:34049)
0810 07:15:07.499650 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.499654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.499666 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:07.499702 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.499704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.499735 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.499797 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.500457 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.500954 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.501791 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.502021 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.502106 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.504277 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.504281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.504284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.504310 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.505348 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.505719 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.505723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.505725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.505799 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.506036 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.506039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.506041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.506205 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.506322 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.506546 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.506651 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:07.509327 15637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:07.512176 15637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:07.526307 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.520466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59476 (local address 127.12.117.130:43429)
0810 07:15:07.521107 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.521120 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.521142 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:15:07.521216 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.521221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.521388 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.521528 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:07.521786 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.522930 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.524637 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.525540 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.525688 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.525819 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.526038 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.526165 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":102,"threads_started":1}
I20260810 07:15:07.526432 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.519995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:43429 (local address 127.0.0.1:59476)
0810 07:15:07.520921 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.520935 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.521030 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.521545 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.521548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.521569 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:07.521681 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.521688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.523361 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.523363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.524503 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.524514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.525685 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.525689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.525808 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.525850 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.525961 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.526075 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":863,"mutex_wait_us":84,"thread_start_us":109,"threads_started":1}
I20260810 07:15:07.526793 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.519858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42919 (local address 127.0.0.1:48900)
0810 07:15:07.520497 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.520544 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.520744 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.521453 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.521459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.521482 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:07.521613 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.521621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.523050 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.523054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.524647 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.524658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.526234 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.526237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.526540 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.526563 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.526634 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.526696 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":115,"thread_start_us":68,"threads_started":1}
I20260810 07:15:07.526804 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.520686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.12.117.131:40889)
0810 07:15:07.521580 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.521591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.521787 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:15:07.521975 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.521979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.522112 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.522235 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:07.522536 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.523448 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.525044 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.525637 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.525853 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.526003 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.526329 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.526673 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":105,"threads_started":1}
I20260810 07:15:07.527220 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.520542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.0.0.1:40042)
0810 07:15:07.521706 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.521718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.521846 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.522230 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.522233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.522250 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:07.522401 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.522407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.523647 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.523650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.524901 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.524911 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.526075 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.526079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.526172 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.526194 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.526264 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.526375 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1111,"mutex_wait_us":207,"thread_start_us":77,"threads_started":1}
I20260810 07:15:07.528014 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.520001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.12.117.129:42919)
0810 07:15:07.520277 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.520292 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.520638 (+ 346us) server_negotiation.cc:407] Connection header received
0810 07:15:07.520925 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.520931 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.521136 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.521265 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:07.521716 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.522936 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.524771 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.525540 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.525691 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.525819 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.527057 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.527158 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260810 07:15:07.529543 16060 tablet_service.cc:1467] Processing CreateTablet for tablet 65aa4c17156243de877d63f03198391d (DEFAULT_TABLE table=test-table [id=1a6d62110f3b419d8a46b958a01ca699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:07.529801 16285 tablet_service.cc:1467] Processing CreateTablet for tablet 65aa4c17156243de877d63f03198391d (DEFAULT_TABLE table=test-table [id=1a6d62110f3b419d8a46b958a01ca699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:07.530092 15840 tablet_service.cc:1467] Processing CreateTablet for tablet 65aa4c17156243de877d63f03198391d (DEFAULT_TABLE table=test-table [id=1a6d62110f3b419d8a46b958a01ca699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:07.530666 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65aa4c17156243de877d63f03198391d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.530673 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65aa4c17156243de877d63f03198391d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.531114 15840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65aa4c17156243de877d63f03198391d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.537253 16429 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Bootstrap starting.
I20260810 07:15:07.538710 16431 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Bootstrap starting.
I20260810 07:15:07.539388 16429 tablet_bootstrap.cc:654] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.539597 16433 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Bootstrap starting.
I20260810 07:15:07.540123 16429 log.cc:824] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.540666 16431 tablet_bootstrap.cc:654] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.541443 16431 log.cc:824] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.541529 16433 tablet_bootstrap.cc:654] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.542074 16429 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: No bootstrap required, opened a new log
I20260810 07:15:07.542275 16429 ts_tablet_manager.cc:1397] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 07:15:07.542308 16433 log.cc:824] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:07.543452 16431 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: No bootstrap required, opened a new log
I20260810 07:15:07.543645 16431 ts_tablet_manager.cc:1397] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:07.544243 16433 tablet_bootstrap.cc:492] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: No bootstrap required, opened a new log
I20260810 07:15:07.544420 16433 ts_tablet_manager.cc:1397] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:07.545435 16429 raft_consensus.cc:348] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.545650 16429 raft_consensus.cc:374] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.545722 16429 raft_consensus.cc:729] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.546130 16429 consensus_queue.cc:260] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.546753 15948 heartbeater.cc:503] Master 127.12.117.190:34049 was elected leader, sending a full tablet report...
I20260810 07:15:07.547075 16429 ts_tablet_manager.cc:1428] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
W20260810 07:15:07.547398 15949 tablet.cc:2369] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:07.547305 16431 raft_consensus.cc:348] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.547547 16431 raft_consensus.cc:374] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.547603 16431 raft_consensus.cc:729] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.548031 16431 consensus_queue.cc:260] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.548642 16163 heartbeater.cc:503] Master 127.12.117.190:34049 was elected leader, sending a full tablet report...
I20260810 07:15:07.548936 16431 ts_tablet_manager.cc:1428] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:15:07.549875 16433 raft_consensus.cc:348] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.550124 16433 raft_consensus.cc:374] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.550207 16433 raft_consensus.cc:729] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.550599 16433 consensus_queue.cc:260] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.551342 16387 heartbeater.cc:503] Master 127.12.117.190:34049 was elected leader, sending a full tablet report...
I20260810 07:15:07.551419 16433 ts_tablet_manager.cc:1428] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260810 07:15:07.723315 16388 tablet.cc:2369] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:07.751626 16164 tablet.cc:2369] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:07.780143 16440 raft_consensus.cc:482] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:07.780344 16440 raft_consensus.cc:504] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.781491 16440 leader_election.cc:288] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:07.785657 16421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.781790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:37361 (local address 127.12.117.129:42919)
0810 07:15:07.781929 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.781933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.782004 (+ 71us) server_negotiation.cc:407] Connection header received
0810 07:15:07.782147 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.782150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.782205 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.782282 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:07.782579 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.783336 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.784565 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.785128 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.785219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.785260 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.785470 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.785540 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:15:07.785660 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.781701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42919 (local address 127.12.117.131:37361)
0810 07:15:07.781893 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.781909 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.782042 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.782320 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.782324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.782345 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:07.782472 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.782480 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.783428 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.783431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.784432 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.784441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.785231 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.785234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.785350 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.785377 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.785452 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.785523 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":19}
I20260810 07:15:07.786091 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.781812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:42481 (local address 127.12.117.130:43429)
0810 07:15:07.781950 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.781955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.782249 (+ 294us) server_negotiation.cc:407] Connection header received
0810 07:15:07.782366 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.782369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.782416 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.782490 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:07.783191 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.783893 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.785032 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.785535 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.785650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.785695 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.785905 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.785980 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:15:07.786103 16441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.781723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:43429 (local address 127.12.117.131:42481)
0810 07:15:07.782133 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.782146 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.782256 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.782511 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.782514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.782531 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:07.783087 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.783097 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.784004 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.784012 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.784919 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.784929 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.785671 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.785675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.785770 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.785793 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.785873 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.785945 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260810 07:15:07.786298 15869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65aa4c17156243de877d63f03198391d" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" is_pre_election: true
I20260810 07:15:07.786588 15869 raft_consensus.cc:2454] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 0.
I20260810 07:15:07.786700 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65aa4c17156243de877d63f03198391d" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d" is_pre_election: true
I20260810 07:15:07.786978 16090 raft_consensus.cc:2454] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 0.
I20260810 07:15:07.787106 16213 leader_election.cc:302] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0df336b3d0145aaadf637bbbdcba656, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:07.787427 16440 raft_consensus.cc:2781] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:07.787531 16440 raft_consensus.cc:482] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:07.787596 16440 raft_consensus.cc:3037] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.788594 16440 raft_consensus.cc:504] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.788906 16440 leader_election.cc:288] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 election: Requested vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:07.789208 15869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65aa4c17156243de877d63f03198391d" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
I20260810 07:15:07.789232 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65aa4c17156243de877d63f03198391d" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d"
I20260810 07:15:07.789345 15869 raft_consensus.cc:3037] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.789345 16090 raft_consensus.cc:3037] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.790526 15869 raft_consensus.cc:2454] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 1.
I20260810 07:15:07.790526 16090 raft_consensus.cc:2454] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 1.
I20260810 07:15:07.790871 16213 leader_election.cc:302] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0df336b3d0145aaadf637bbbdcba656, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:07.791078 16440 raft_consensus.cc:2781] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.791312 16440 raft_consensus.cc:686] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 1 LEADER]: Becoming Leader. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Running, Role: LEADER
I20260810 07:15:07.791630 16440 consensus_queue.cc:237] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.794450 15637 catalog_manager.cc:5495] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } health_report { overall_health: HEALTHY } } }
Aug 10 07:15:07 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346106, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:07.827901 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.820504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34049 (local address 127.0.0.1:52634)
0810 07:15:07.820757 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.820770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.820855 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.821045 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.821047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.821384 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:07.821469 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.821475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.822156 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.822158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.822736 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.822745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.823100 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.823103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.823175 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.823727 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:07.823739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:07.825038 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:07.826591 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.826599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.826604 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.826821 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.827036 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:07.827038 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:07.827039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:07.827110 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:07.827459 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:07.827464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:07.827585 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.827672 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.827784 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260810 07:15:07.827975 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.820548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.12.117.190:34049)
0810 07:15:07.820655 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.820659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.820830 (+ 171us) server_negotiation.cc:407] Connection header received
0810 07:15:07.820929 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.820931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.820968 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.821026 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:07.821554 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.822062 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.822821 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.823024 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.823089 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.825168 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:07.825171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:07.825173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:07.825195 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:07.826483 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.826894 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.826897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.826898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.826971 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:07.827171 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:07.827173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:07.827174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:07.827374 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:07.827447 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.827713 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.827847 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":37}
I20260810 07:15:07.831173 15637 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:52634:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:15:07.831848 15637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:07.836114 15840 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebef735a87487fb6fe88e0a3463783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeda7712b390496a8ee552a7778828f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:07.836205 16285 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebef735a87487fb6fe88e0a3463783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeda7712b390496a8ee552a7778828f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:07.836163 16060 tablet_service.cc:1467] Processing CreateTablet for tablet e1ebef735a87487fb6fe88e0a3463783 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aeda7712b390496a8ee552a7778828f0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:07.836791 15840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebef735a87487fb6fe88e0a3463783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.836791 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebef735a87487fb6fe88e0a3463783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.836802 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1ebef735a87487fb6fe88e0a3463783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:07.839175 16431 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: Bootstrap starting.
I20260810 07:15:07.839478 16433 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: Bootstrap starting.
I20260810 07:15:07.840039 16431 tablet_bootstrap.cc:654] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.840273 16433 tablet_bootstrap.cc:654] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.840859 16429 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: Bootstrap starting.
I20260810 07:15:07.841105 16431 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: No bootstrap required, opened a new log
I20260810 07:15:07.841193 16431 ts_tablet_manager.cc:1397] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:07.841396 16433 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: No bootstrap required, opened a new log
I20260810 07:15:07.841466 16433 ts_tablet_manager.cc:1397] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:07.841635 16429 tablet_bootstrap.cc:654] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:07.841653 16431 raft_consensus.cc:348] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.841750 16431 raft_consensus.cc:374] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.841781 16431 raft_consensus.cc:729] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.841861 16433 raft_consensus.cc:348] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.841918 16431 consensus_queue.cc:260] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.841965 16433 raft_consensus.cc:374] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.841993 16433 raft_consensus.cc:729] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.842113 16433 consensus_queue.cc:260] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.842190 16431 ts_tablet_manager.cc:1428] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:07.842361 16433 ts_tablet_manager.cc:1428] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:07.842393 16440 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.842502 16440 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.842199 16439 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.843122 16439 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.843238 16429 tablet_bootstrap.cc:492] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: No bootstrap required, opened a new log
I20260810 07:15:07.843336 16429 ts_tablet_manager.cc:1397] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 07:15:07.843768 16429 raft_consensus.cc:348] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.843873 16429 raft_consensus.cc:374] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:07.843900 16429 raft_consensus.cc:729] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:07.844017 16429 consensus_queue.cc:260] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.844300 16429 ts_tablet_manager.cc:1428] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:07.844481 16438 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.844614 16438 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.913470 16440 raft_consensus.cc:482] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:07.913616 16440 raft_consensus.cc:504] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.913997 16440 leader_election.cc:288] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:07.914389 15869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebef735a87487fb6fe88e0a3463783" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" is_pre_election: true
I20260810 07:15:07.914412 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebef735a87487fb6fe88e0a3463783" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d" is_pre_election: true
I20260810 07:15:07.914536 16090 raft_consensus.cc:2454] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 0.
I20260810 07:15:07.914542 15869 raft_consensus.cc:2454] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 0.
I20260810 07:15:07.914860 16212 leader_election.cc:302] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, b0df336b3d0145aaadf637bbbdcba656; no voters:
I20260810 07:15:07.915051 16440 raft_consensus.cc:2781] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:07.915128 16440 raft_consensus.cc:482] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:07.915166 16440 raft_consensus.cc:3037] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.916203 16440 raft_consensus.cc:504] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.916550 16440 leader_election.cc:288] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 election: Requested vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:07.916911 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebef735a87487fb6fe88e0a3463783" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d"
I20260810 07:15:07.917021 16090 raft_consensus.cc:3037] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.917922 16090 raft_consensus.cc:2454] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 1.
I20260810 07:15:07.918159 15869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1ebef735a87487fb6fe88e0a3463783" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
I20260810 07:15:07.918176 16212 leader_election.cc:302] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, b0df336b3d0145aaadf637bbbdcba656; no voters:
I20260810 07:15:07.918265 15869 raft_consensus.cc:3037] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:07.918357 16440 raft_consensus.cc:2781] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:07.918459 16440 raft_consensus.cc:686] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [term 1 LEADER]: Becoming Leader. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Running, Role: LEADER
I20260810 07:15:07.918598 16440 consensus_queue.cc:237] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:07.919226 16440 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: TxnStatusTablet state changed. Reason: New leader b0df336b3d0145aaadf637bbbdcba656. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.919348 16440 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:07.919329 15869 raft_consensus.cc:2454] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0df336b3d0145aaadf637bbbdcba656 in term 1.
I20260810 07:15:07.919791 16461 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:15:07.920001 16461 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:15:07.920496 15636 catalog_manager.cc:5495] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } health_report { overall_health: HEALTHY } } }
I20260810 07:15:07.941506 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.937956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.0.0.1:40050)
0810 07:15:07.938107 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.938123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.938228 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.938549 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.938552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.938559 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:07.938659 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.938664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.939393 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.939397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.940014 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.940023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.940467 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.940470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.940559 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.940603 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:07.941215 (+ 612us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:07.941231 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.941294 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.941380 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 07:15:07.941506 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.938030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.12.117.131:40889)
0810 07:15:07.938151 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.938155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.938169 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:07.938384 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.938387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.938440 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.938522 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:07.938756 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.939254 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.940127 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.940355 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.940452 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.940737 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:07.941117 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:07.941218 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.941306 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.941363 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:15:07.946604 16090 raft_consensus.cc:1261] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Refusing update from remote peer b0df336b3d0145aaadf637bbbdcba656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:07.946622 15869 raft_consensus.cc:1261] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Refusing update from remote peer b0df336b3d0145aaadf637bbbdcba656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:07.947291 16440 consensus_queue.cc:1035] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:07.947477 16444 consensus_queue.cc:1035] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:07.951537 16439 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: TxnStatusTablet state changed. Reason: New leader b0df336b3d0145aaadf637bbbdcba656. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.951650 16439 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.954641 16438 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: TxnStatusTablet state changed. Reason: New leader b0df336b3d0145aaadf637bbbdcba656. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.954766 16438 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.956223 16440 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.956338 16440 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:07.957806 16438 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.957907 16438 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P f5cbc53ecf1248b6b133cb0fd089b4ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.958455 16469 mvcc.cc:205] Tried to move back new op lower bound from 7316873658143174656 to 7316873658035875840. Current Snapshot: MvccSnapshot[applied={T|T < 7316873658143174656}]
I20260810 07:15:07.958504 16466 mvcc.cc:205] Tried to move back new op lower bound from 7316873658143174656 to 7316873658035875840. Current Snapshot: MvccSnapshot[applied={T|T < 7316873658143174656 or (T in {7316873658143174656})}]
I20260810 07:15:07.958957 16444 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.959067 16444 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P b0df336b3d0145aaadf637bbbdcba656: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:07.960144 16439 tablet_replica.cc:439] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b0df336b3d0145aaadf637bbbdcba656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } } }
I20260810 07:15:07.960246 16439 tablet_replica.cc:442] T e1ebef735a87487fb6fe88e0a3463783 P 9f130ee7d2ae402f992256590408d94d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:07.960475 16467 mvcc.cc:205] Tried to move back new op lower bound from 7316873658143174656 to 7316873658035875840. Current Snapshot: MvccSnapshot[applied={T|T < 7316873658143174656}]
I20260810 07:15:07.968300 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.965049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.12.117.131:40889)
0810 07:15:07.965178 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.965181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.965213 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:15:07.965340 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.965342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.965377 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.965432 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:07.965680 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.966168 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.967387 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.967623 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.967695 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.967948 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:07.968010 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:07.968055 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.968159 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.968198 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:15:07.978031 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.975078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:56357 (local address 127.12.117.131:40889)
0810 07:15:07.975170 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:07.975174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:07.975185 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:07.975287 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:07.975290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:07.975329 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:07.975385 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:07.975637 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.976130 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.977039 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.977231 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.977313 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.977592 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:07.977651 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:07.977707 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:07.977794 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:15:07.977857 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":38}
I20260810 07:15:07.978031 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:07.974973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.12.117.131:56357)
0810 07:15:07.975116 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:07.975131 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:07.975202 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:07.975411 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:07.975414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:07.975421 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:07.975548 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.975554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.976243 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.976246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.976926 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:07.976936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:07.977326 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:07.977329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:07.977418 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:07.977457 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:07.977724 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:07.977729 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:07.977785 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:15:07.977865 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260810 07:15:07.984849 16090 raft_consensus.cc:1261] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Refusing update from remote peer b0df336b3d0145aaadf637bbbdcba656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:07.984962 15869 raft_consensus.cc:1261] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Refusing update from remote peer b0df336b3d0145aaadf637bbbdcba656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:07.985242 16440 consensus_queue.cc:1035] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:07.985430 16444 consensus_queue.cc:1035] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:08.028841 15636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:08.029002 15636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:08.031311 15636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Sending DeleteTablet for 3 replicas of tablet 65aa4c17156243de877d63f03198391d
I20260810 07:15:08.032022 15840 tablet_service.cc:1514] Processing DeleteTablet for tablet 65aa4c17156243de877d63f03198391d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:08 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:48900
I20260810 07:15:08.032011 16060 tablet_service.cc:1514] Processing DeleteTablet for tablet 65aa4c17156243de877d63f03198391d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:08 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59476
I20260810 07:15:08.032517 16496 tablet_replica.cc:330] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: stopping tablet replica
I20260810 07:15:08.032589 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet 65aa4c17156243de877d63f03198391d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:08 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:40042
I20260810 07:15:08.032872 16496 raft_consensus.cc:2229] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:08.033102 16498 tablet_replica.cc:330] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: stopping tablet replica
I20260810 07:15:08.033210 16497 tablet_replica.cc:330] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: stopping tablet replica
I20260810 07:15:08.033223 15636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:08.033418 16498 raft_consensus.cc:2229] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:08.033470 16497 raft_consensus.cc:2229] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 07:15:08.033620 15636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:08.033643 16497 raft_consensus.cc:2258] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:08.033746 16498 raft_consensus.cc:2258] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:08.034621 16496 raft_consensus.cc:2258] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:08.035594 16498 ts_tablet_manager.cc:1905] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:08.036760 16497 ts_tablet_manager.cc:1905] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:08.036901 16496 ts_tablet_manager.cc:1905] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:08.038460 16285 tablet_service.cc:1467] Processing CreateTablet for tablet fb13143f858e4f16946a40bcfe0198fd (DEFAULT_TABLE table=test-table [id=046bb57d9e714273bf6a5e6b48d363c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:08.038483 16060 tablet_service.cc:1467] Processing CreateTablet for tablet fb13143f858e4f16946a40bcfe0198fd (DEFAULT_TABLE table=test-table [id=046bb57d9e714273bf6a5e6b48d363c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:08.038736 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb13143f858e4f16946a40bcfe0198fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:08.038843 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb13143f858e4f16946a40bcfe0198fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:08.039506 15840 tablet_service.cc:1467] Processing CreateTablet for tablet fb13143f858e4f16946a40bcfe0198fd (DEFAULT_TABLE table=test-table [id=046bb57d9e714273bf6a5e6b48d363c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:08.039726 15840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb13143f858e4f16946a40bcfe0198fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:08.041116 16431 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Bootstrap starting.
I20260810 07:15:08.041960 16429 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Bootstrap starting.
I20260810 07:15:08.042057 16431 tablet_bootstrap.cc:654] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:08.042882 16429 tablet_bootstrap.cc:654] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:08.044170 16433 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Bootstrap starting.
I20260810 07:15:08.044644 16498 ts_tablet_manager.cc:1918] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:08.044734 16498 log.cc:1196] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/65aa4c17156243de877d63f03198391d
I20260810 07:15:08.045162 16496 ts_tablet_manager.cc:1918] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:08.045255 16496 log.cc:1196] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/65aa4c17156243de877d63f03198391d
I20260810 07:15:08.045428 16433 tablet_bootstrap.cc:654] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:08.045569 16496 ts_tablet_manager.cc:1939] T 65aa4c17156243de877d63f03198391d P 9f130ee7d2ae402f992256590408d94d: Deleting consensus metadata
I20260810 07:15:08.046691 16433 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: No bootstrap required, opened a new log
I20260810 07:15:08.046769 16433 ts_tablet_manager.cc:1397] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:08.046808 15615 catalog_manager.cc:4841] TS 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429): tablet 65aa4c17156243de877d63f03198391d (table test-table [id=1a6d62110f3b419d8a46b958a01ca699]) successfully deleted
I20260810 07:15:08.047209 16498 ts_tablet_manager.cc:1939] T 65aa4c17156243de877d63f03198391d P b0df336b3d0145aaadf637bbbdcba656: Deleting consensus metadata
I20260810 07:15:08.047173 16433 raft_consensus.cc:348] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.047295 16433 raft_consensus.cc:374] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:08.047326 16433 raft_consensus.cc:729] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Initialized, Role: FOLLOWER
I20260810 07:15:08.047434 16433 consensus_queue.cc:260] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.047665 16433 ts_tablet_manager.cc:1428] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:08.048290 15616 catalog_manager.cc:4841] TS b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889): tablet 65aa4c17156243de877d63f03198391d (table test-table [id=1a6d62110f3b419d8a46b958a01ca699]) successfully deleted
I20260810 07:15:08.049280 16429 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: No bootstrap required, opened a new log
I20260810 07:15:08.049368 16429 ts_tablet_manager.cc:1397] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260810 07:15:08.049794 16429 raft_consensus.cc:348] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.049901 16429 raft_consensus.cc:374] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:08.049933 16429 raft_consensus.cc:729] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:08.050074 16429 consensus_queue.cc:260] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.050338 16429 ts_tablet_manager.cc:1428] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:08.057538 16497 ts_tablet_manager.cc:1918] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:08.057708 16497 log.cc:1196] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/65aa4c17156243de877d63f03198391d
I20260810 07:15:08.058095 16497 ts_tablet_manager.cc:1939] T 65aa4c17156243de877d63f03198391d P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting consensus metadata
I20260810 07:15:08.059365 15616 catalog_manager.cc:4841] TS f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919): tablet 65aa4c17156243de877d63f03198391d (table test-table [id=1a6d62110f3b419d8a46b958a01ca699]) successfully deleted
I20260810 07:15:08.059993 16431 tablet_bootstrap.cc:492] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: No bootstrap required, opened a new log
I20260810 07:15:08.060074 16431 ts_tablet_manager.cc:1397] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Time spent bootstrapping tablet: real 0.019s user 0.002s sys 0.000s
I20260810 07:15:08.060459 16431 raft_consensus.cc:348] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.060564 16431 raft_consensus.cc:374] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:08.060590 16431 raft_consensus.cc:729] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Initialized, Role: FOLLOWER
I20260810 07:15:08.060698 16431 consensus_queue.cc:260] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.060916 16431 ts_tablet_manager.cc:1428] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:08.255826 16438 raft_consensus.cc:482] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:08.255959 16438 raft_consensus.cc:504] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.256999 16438 leader_election.cc:288] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:08.261896 16510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.257244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.12.117.129:59747)
0810 07:15:08.257623 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:08.257634 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:08.257703 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:08.257950 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:08.257953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:08.257973 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:08.258078 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.258085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.259109 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.259113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.260460 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.260472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.261355 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.261359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.261508 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.261532 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:15:08.261646 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 07:15:08.261724 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":98,"threads_started":1}
I20260810 07:15:08.261960 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.257608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59747 (local address 127.12.117.131:40889)
0810 07:15:08.257713 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:08.257717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:08.257755 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:15:08.257805 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:08.257808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:08.257848 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:08.257924 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:08.258204 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.258958 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.260610 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.261229 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.261349 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.261408 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:08.261690 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 07:15:08.261793 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:15:08.262265 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.257321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:60155 (local address 127.12.117.130:43429)
0810 07:15:08.257438 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:08.257443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:08.257594 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:15:08.257796 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:08.257800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:08.257838 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:08.257875 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:08.258271 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.258765 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.260965 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.261597 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.261710 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.261757 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:08.262071 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 07:15:08.262150 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:08.262607 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.257206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:43429 (local address 127.12.117.129:60155)
0810 07:15:08.257504 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:08.257518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:08.257673 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:08.258033 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:08.258036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:08.258050 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:08.258128 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.258134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.259689 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.259695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.260827 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.260840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.261816 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.261820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.261936 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.261960 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:15:08.262395 (+ 435us) client_negotiation.cc:240] Negotiation successful
0810 07:15:08.262474 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":86,"threads_started":1}
I20260810 07:15:08.263088 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d" is_pre_election: true
I20260810 07:15:08.263236 16090 raft_consensus.cc:2454] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5cbc53ecf1248b6b133cb0fd089b4ad in term 0.
I20260810 07:15:08.264078 15774 leader_election.cc:302] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:08.264364 16438 raft_consensus.cc:2781] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:08.264436 16438 raft_consensus.cc:482] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:08.264473 16438 raft_consensus.cc:3037] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:08.266091 16438 raft_consensus.cc:504] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.266243 16490 raft_consensus.cc:482] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:08.266328 16490 raft_consensus.cc:504] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.266467 16438 leader_election.cc:288] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [CANDIDATE]: Term 1 election: Requested vote from peers 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:08.266683 16490 leader_election.cc:288] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429)
I20260810 07:15:08.266873 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d"
I20260810 07:15:08.266999 16090 raft_consensus.cc:3037] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:08.267336 16311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656" is_pre_election: true
I20260810 07:15:08.267544 16311 raft_consensus.cc:2454] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5cbc53ecf1248b6b133cb0fd089b4ad in term 0.
I20260810 07:15:08.267797 16310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656"
I20260810 07:15:08.267902 16310 raft_consensus.cc:3037] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:08.268196 16090 raft_consensus.cc:2454] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5cbc53ecf1248b6b133cb0fd089b4ad in term 1.
I20260810 07:15:08.268644 15774 leader_election.cc:302] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:08.268827 16438 raft_consensus.cc:2781] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:08.269057 16310 raft_consensus.cc:2454] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5cbc53ecf1248b6b133cb0fd089b4ad in term 1.
I20260810 07:15:08.269054 16438 raft_consensus.cc:686] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 LEADER]: Becoming Leader. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Running, Role: LEADER
I20260810 07:15:08.269305 16438 consensus_queue.cc:237] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:08.273372 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" is_pre_election: true
I20260810 07:15:08.273648 15636 catalog_manager.cc:5495] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad reported cstate change: term changed from 0 to 1, leader changed from <none> to f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129). New cstate: current_term: 1 leader_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:08.275415 16090 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb13143f858e4f16946a40bcfe0198fd" candidate_uuid: "b0df336b3d0145aaadf637bbbdcba656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f130ee7d2ae402f992256590408d94d" is_pre_election: true
I20260810 07:15:08.275604 16090 raft_consensus.cc:2379] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0df336b3d0145aaadf637bbbdcba656 in current term 1: Already voted for candidate f5cbc53ecf1248b6b133cb0fd089b4ad in this term.
I20260810 07:15:08.275950 16212 leader_election.cc:302] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0df336b3d0145aaadf637bbbdcba656; no voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad
I20260810 07:15:08.276153 16490 raft_consensus.cc:2726] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:15:08.295390 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.289738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.12.117.129:42919)
0810 07:15:08.289972 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:08.289977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:08.290003 (+ 26us) server_negotiation.cc:407] Connection header received
0810 07:15:08.290145 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:08.290148 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:08.290216 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:08.290344 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:08.290586 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.291444 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.292921 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.293359 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.293785 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.293886 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:08.294617 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:08.295107 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:08.295148 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:15:08.295210 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260810 07:15:08.306372 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.302921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:36017 (local address 127.12.117.131:40889)
0810 07:15:08.303054 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:08.303059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:08.303225 (+ 166us) server_negotiation.cc:407] Connection header received
0810 07:15:08.303410 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:08.303412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:08.303455 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:08.303514 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:08.303790 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.304512 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.305396 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.305614 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.305698 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.305984 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:08.306055 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:08.306108 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:08.306199 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 07:15:08.306245 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:08.306434 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.302847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.12.117.129:36017)
0810 07:15:08.303147 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:08.303163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:08.303312 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:08.303556 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:08.303559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:08.303568 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:08.303694 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.303701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.304625 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.304629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.305273 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.305284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.305713 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.305716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.305811 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.305853 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:08.306126 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:08.306133 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:08.306190 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 07:15:08.306269 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260810 07:15:08.312250 16090 raft_consensus.cc:1261] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Refusing update from remote peer f5cbc53ecf1248b6b133cb0fd089b4ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:08.312763 16438 consensus_queue.cc:1035] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:08.313879 16310 raft_consensus.cc:1261] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Refusing update from remote peer f5cbc53ecf1248b6b133cb0fd089b4ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:08.314550 16512 consensus_queue.cc:1035] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:08.330025 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.327107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35607 (local address 127.12.117.129:42919)
0810 07:15:08.327206 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:08.327210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:08.327256 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:15:08.327426 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:08.327429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:08.327473 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:08.327528 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:08.327760 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.328416 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.329210 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.329414 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.329483 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.329645 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:08.329704 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:08.329749 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:08.329853 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 07:15:08.329901 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:08.330040 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:08.327068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42919 (local address 127.12.117.131:35607)
0810 07:15:08.327195 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:08.327211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:08.327318 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:08.327546 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:08.327549 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:08.327556 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:08.327644 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.327650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.328508 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.328511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.329119 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:08.329129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:08.329488 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:08.329490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:08.329546 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:08.329571 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:08.329769 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:08.329775 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:08.329845 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:15:08.329927 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:15:08.330947 16469 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:15:09.895293 15637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:09.895447 15637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:09.897640 15637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Sending DeleteTablet for 3 replicas of tablet fb13143f858e4f16946a40bcfe0198fd
I20260810 07:15:09.898181 15840 tablet_service.cc:1514] Processing DeleteTablet for tablet fb13143f858e4f16946a40bcfe0198fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:09 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:48900
I20260810 07:15:09.898238 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet fb13143f858e4f16946a40bcfe0198fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:09 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:40042
I20260810 07:15:09.898350 16060 tablet_service.cc:1514] Processing DeleteTablet for tablet fb13143f858e4f16946a40bcfe0198fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:09 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59476
I20260810 07:15:09.898581 16567 tablet_replica.cc:330] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: stopping tablet replica
I20260810 07:15:09.898654 16566 tablet_replica.cc:330] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: stopping tablet replica
I20260810 07:15:09.898710 16567 raft_consensus.cc:2229] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:09.898783 16566 raft_consensus.cc:2229] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:09.898823 16567 raft_consensus.cc:2258] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:09.899101 16566 raft_consensus.cc:2258] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:09.899361 16568 tablet_replica.cc:330] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: stopping tablet replica
I20260810 07:15:09.899526 16568 raft_consensus.cc:2229] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:09.899580 16567 ts_tablet_manager.cc:1905] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:09.899706 16568 raft_consensus.cc:2258] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:09.900342 16568 ts_tablet_manager.cc:1905] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:09.900992 16566 ts_tablet_manager.cc:1905] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:09.902406 16567 ts_tablet_manager.cc:1918] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:15:09.902531 16567 log.cc:1196] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/fb13143f858e4f16946a40bcfe0198fd
I20260810 07:15:09.902860 16567 ts_tablet_manager.cc:1939] T fb13143f858e4f16946a40bcfe0198fd P b0df336b3d0145aaadf637bbbdcba656: Deleting consensus metadata
I20260810 07:15:09.903538 15616 catalog_manager.cc:4841] TS b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889): tablet fb13143f858e4f16946a40bcfe0198fd (table test-table [id=046bb57d9e714273bf6a5e6b48d363c1]) successfully deleted
I20260810 07:15:09.903632 16568 ts_tablet_manager.cc:1918] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:15:09.903704 16568 log.cc:1196] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/fb13143f858e4f16946a40bcfe0198fd
I20260810 07:15:09.903924 16568 ts_tablet_manager.cc:1939] T fb13143f858e4f16946a40bcfe0198fd P 9f130ee7d2ae402f992256590408d94d: Deleting consensus metadata
I20260810 07:15:09.904523 15615 catalog_manager.cc:4841] TS 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429): tablet fb13143f858e4f16946a40bcfe0198fd (table test-table [id=046bb57d9e714273bf6a5e6b48d363c1]) successfully deleted
I20260810 07:15:09.904636 16566 ts_tablet_manager.cc:1918] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:15:09.904709 16566 log.cc:1196] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/fb13143f858e4f16946a40bcfe0198fd
I20260810 07:15:09.904942 16566 ts_tablet_manager.cc:1939] T fb13143f858e4f16946a40bcfe0198fd P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting consensus metadata
I20260810 07:15:09.905486 15616 catalog_manager.cc:4841] TS f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919): tablet fb13143f858e4f16946a40bcfe0198fd (table test-table [id=046bb57d9e714273bf6a5e6b48d363c1]) successfully deleted
Aug 10 07:15:09 dist-test-slave-gjrz krb5kdc[15561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:09 dist-test-slave-gjrz krb5kdc[15561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346109, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:09.930241 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:09.919292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41404 (local address 127.12.117.190:34049)
0810 07:15:09.919549 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:09.919557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:09.919572 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:09.919711 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:09.919714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:09.919763 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:09.919835 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:09.920517 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.921064 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.921864 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:09.922154 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:09.922245 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:09.927574 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:09.927578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:09.927580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:09.927605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:09.928838 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.929233 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.929236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.929237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.929321 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:09.929542 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:09.929545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:09.929546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:09.929729 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:09.929831 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:09.930015 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:15:09.930116 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":95,"threads_started":1}
I20260810 07:15:09.932462 15637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:09.932879 15637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:09.937000 15840 tablet_service.cc:1467] Processing CreateTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac (DEFAULT_TABLE table=test-table [id=94e9cb5b9f8944aaae466522d37238fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:09.937075 16060 tablet_service.cc:1467] Processing CreateTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac (DEFAULT_TABLE table=test-table [id=94e9cb5b9f8944aaae466522d37238fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:09.937168 16285 tablet_service.cc:1467] Processing CreateTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac (DEFAULT_TABLE table=test-table [id=94e9cb5b9f8944aaae466522d37238fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:09.937282 15840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cd177bb89c4e4f80c5ac14042850ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:09.937345 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cd177bb89c4e4f80c5ac14042850ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:09.937381 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32cd177bb89c4e4f80c5ac14042850ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:09.939129 16580 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Bootstrap starting.
I20260810 07:15:09.939669 16583 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Bootstrap starting.
I20260810 07:15:09.939950 16580 tablet_bootstrap.cc:654] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:09.940047 16582 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Bootstrap starting.
I20260810 07:15:09.940908 16583 tablet_bootstrap.cc:654] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:09.940955 16582 tablet_bootstrap.cc:654] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:09.941254 16580 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: No bootstrap required, opened a new log
I20260810 07:15:09.941345 16580 ts_tablet_manager.cc:1397] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:09.941708 16580 raft_consensus.cc:348] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.941830 16580 raft_consensus.cc:374] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:09.941884 16580 raft_consensus.cc:729] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:09.942081 16583 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: No bootstrap required, opened a new log
I20260810 07:15:09.942015 16580 consensus_queue.cc:260] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.942312 16580 ts_tablet_manager.cc:1428] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:09.942425 16583 ts_tablet_manager.cc:1397] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:09.942942 16583 raft_consensus.cc:348] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.943061 16583 raft_consensus.cc:374] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:09.943095 16583 raft_consensus.cc:729] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Initialized, Role: FOLLOWER
I20260810 07:15:09.943118 16582 tablet_bootstrap.cc:492] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: No bootstrap required, opened a new log
I20260810 07:15:09.943194 16582 ts_tablet_manager.cc:1397] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:15:09.943346 16583 consensus_queue.cc:260] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.943624 16582 raft_consensus.cc:348] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.943737 16583 ts_tablet_manager.cc:1428] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:09.943739 16582 raft_consensus.cc:374] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:09.943776 16582 raft_consensus.cc:729] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Initialized, Role: FOLLOWER
I20260810 07:15:09.943907 16582 consensus_queue.cc:260] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:09.944167 16582 ts_tablet_manager.cc:1428] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:10.310447 16589 raft_consensus.cc:482] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:10.310585 16589 raft_consensus.cc:504] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.311687 16589 leader_election.cc:288] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:10.316023 16592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.311985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:39307 (local address 127.12.117.129:42919)
0810 07:15:10.312294 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.312304 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.312333 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:15:10.312395 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.312399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.312463 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.312554 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:10.312872 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.313393 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.314921 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.315399 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.315513 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.315570 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.315795 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.315885 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":104,"threads_started":1}
I20260810 07:15:10.315977 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.311864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42919 (local address 127.12.117.130:39307)
0810 07:15:10.312186 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:10.312200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:10.312299 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:10.312621 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:10.312624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:10.312645 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:10.312760 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.312765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.313515 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.313518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.314791 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.314803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.315513 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.315517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.315649 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.315675 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:15:10.315744 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:10.315821 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":3,"thread_start_us":136,"threads_started":1}
I20260810 07:15:10.316032 16593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.312314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.12.117.130:43767)
0810 07:15:10.312629 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:10.312641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:10.312739 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:10.312976 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:10.312979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:10.312996 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:10.313121 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.313127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.313889 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.313893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.315000 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.315009 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.315687 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.315690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.315779 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.315798 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:15:10.315862 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:15:10.315919 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":25,"thread_start_us":101,"threads_started":1}
I20260810 07:15:10.316326 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.312437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:43767 (local address 127.12.117.131:40889)
0810 07:15:10.312721 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.312730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.312745 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:10.312839 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.312842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.312887 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.312957 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:10.313229 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.313764 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.315111 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.315590 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.315706 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.315756 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.315881 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.316000 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":106,"threads_started":1}
I20260810 07:15:10.316579 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cd177bb89c4e4f80c5ac14042850ac" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" is_pre_election: true
I20260810 07:15:10.316651 16310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cd177bb89c4e4f80c5ac14042850ac" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656" is_pre_election: true
I20260810 07:15:10.316741 15868 raft_consensus.cc:2454] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 0.
I20260810 07:15:10.316780 16310 raft_consensus.cc:2454] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 0.
I20260810 07:15:10.317131 15989 leader_election.cc:302] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:10.317374 16589 raft_consensus.cc:2781] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:10.317444 16589 raft_consensus.cc:482] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:10.317488 16589 raft_consensus.cc:3037] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.318466 16589 raft_consensus.cc:504] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.318763 16589 leader_election.cc:288] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 election: Requested vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:10.319118 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cd177bb89c4e4f80c5ac14042850ac" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
I20260810 07:15:10.319134 16310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32cd177bb89c4e4f80c5ac14042850ac" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656"
I20260810 07:15:10.319221 15868 raft_consensus.cc:3037] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.319250 16310 raft_consensus.cc:3037] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.320130 15868 raft_consensus.cc:2454] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 1.
I20260810 07:15:10.320183 16310 raft_consensus.cc:2454] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 1.
I20260810 07:15:10.320384 15989 leader_election.cc:302] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:10.320551 16589 raft_consensus.cc:2781] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:10.320758 16589 raft_consensus.cc:686] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 1 LEADER]: Becoming Leader. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Running, Role: LEADER
I20260810 07:15:10.320995 16589 consensus_queue.cc:237] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.323323 15637 catalog_manager.cc:5495] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f130ee7d2ae402f992256590408d94d (127.12.117.130). New cstate: current_term: 1 leader_uuid: "9f130ee7d2ae402f992256590408d94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:10.397424 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.393794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.12.117.130:43429)
0810 07:15:10.394043 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.394050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.394062 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:10.394131 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.394132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.394184 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.394293 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:10.394526 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.395088 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.395974 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.396215 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.396303 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.396578 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:10.397044 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:10.397151 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.397224 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.397275 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
I20260810 07:15:10.406597 16594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.403652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:52849 (local address 127.12.117.131:40889)
0810 07:15:10.403750 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.403753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.403876 (+ 123us) server_negotiation.cc:407] Connection header received
0810 07:15:10.403995 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.403997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.404037 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.404101 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:10.404283 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.404752 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.405517 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.405768 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.405863 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.406111 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:10.406170 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:10.406221 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.406374 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.406418 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:10.406598 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.403498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40889 (local address 127.12.117.130:52849)
0810 07:15:10.403798 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:10.403814 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:10.403917 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:10.404114 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:10.404116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:10.404122 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:10.404211 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.404215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.404828 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.404830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.405405 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.405413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.405880 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.405884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.405964 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.406007 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:10.406272 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:10.406280 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 07:15:10.406346 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:15:10.406435 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
I20260810 07:15:10.412123 15868 raft_consensus.cc:1261] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Refusing update from remote peer 9f130ee7d2ae402f992256590408d94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:10.412166 16310 raft_consensus.cc:1261] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Refusing update from remote peer 9f130ee7d2ae402f992256590408d94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:10.412536 16595 consensus_queue.cc:1035] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:10.412724 16589 consensus_queue.cc:1035] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:10.429030 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.425535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:47607 (local address 127.12.117.130:43429)
0810 07:15:10.425636 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:10.425641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:10.425919 (+ 278us) server_negotiation.cc:407] Connection header received
0810 07:15:10.426075 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:10.426077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:10.426126 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:10.426199 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:10.426520 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.427038 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.427909 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.428181 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.428290 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.428620 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:10.428688 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:10.428743 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:10.428859 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:15:10.428911 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:10.429085 16621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:10.425493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:43429 (local address 127.12.117.131:47607)
0810 07:15:10.425825 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:10.425838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:10.425951 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:10.426300 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:10.426303 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:10.426312 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:10.426424 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.426431 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.427126 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.427129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.427805 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:10.427817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:10.428375 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:10.428379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:10.428471 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:10.428507 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:10.428766 (+ 259us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:10.428773 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:10.428839 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:15:10.428920 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":90,"threads_started":1}
I20260810 07:15:10.450726 15637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:10.450881 15637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:10.452759 15637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Sending DeleteTablet for 3 replicas of tablet 32cd177bb89c4e4f80c5ac14042850ac
I20260810 07:15:10.453280 16060 tablet_service.cc:1514] Processing DeleteTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:10 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59476
I20260810 07:15:10.453405 15840 tablet_service.cc:1514] Processing DeleteTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:10 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:48900
I20260810 07:15:10.453635 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet 32cd177bb89c4e4f80c5ac14042850ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:10 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:40042
I20260810 07:15:10.453680 16629 tablet_replica.cc:330] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: stopping tablet replica
I20260810 07:15:10.453791 16629 raft_consensus.cc:2229] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:10.453886 16629 raft_consensus.cc:2258] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:10.453874 15637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:10.454057 16627 tablet_replica.cc:330] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: stopping tablet replica
I20260810 07:15:10.454159 16627 raft_consensus.cc:2229] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 1 LEADER]: Raft consensus shutting down.
W20260810 07:15:10.454258 15637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:10.454425 16627 raft_consensus.cc:2258] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:10.454540 16629 ts_tablet_manager.cc:1905] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:10.454685 16631 tablet_replica.cc:330] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: stopping tablet replica
I20260810 07:15:10.454802 16631 raft_consensus.cc:2229] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:10.454893 16631 raft_consensus.cc:2258] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:10.455358 16627 ts_tablet_manager.cc:1905] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:10.455535 16631 ts_tablet_manager.cc:1905] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:10.457167 16629 ts_tablet_manager.cc:1918] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:10.457255 16629 log.cc:1196] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/32cd177bb89c4e4f80c5ac14042850ac
I20260810 07:15:10.457508 16629 ts_tablet_manager.cc:1939] T 32cd177bb89c4e4f80c5ac14042850ac P f5cbc53ecf1248b6b133cb0fd089b4ad: Deleting consensus metadata
I20260810 07:15:10.458138 15616 catalog_manager.cc:4841] TS f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919): tablet 32cd177bb89c4e4f80c5ac14042850ac (table test-table [id=94e9cb5b9f8944aaae466522d37238fe]) successfully deleted
I20260810 07:15:10.458195 16631 ts_tablet_manager.cc:1918] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:10.458271 16631 log.cc:1196] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/32cd177bb89c4e4f80c5ac14042850ac
I20260810 07:15:10.458384 16627 ts_tablet_manager.cc:1918] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:10.458521 16627 log.cc:1196] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/32cd177bb89c4e4f80c5ac14042850ac
I20260810 07:15:10.458523 16631 ts_tablet_manager.cc:1939] T 32cd177bb89c4e4f80c5ac14042850ac P b0df336b3d0145aaadf637bbbdcba656: Deleting consensus metadata
I20260810 07:15:10.458841 16627 ts_tablet_manager.cc:1939] T 32cd177bb89c4e4f80c5ac14042850ac P 9f130ee7d2ae402f992256590408d94d: Deleting consensus metadata
I20260810 07:15:10.459097 16060 tablet_service.cc:1467] Processing CreateTablet for tablet d4243b45f7144cea8396e842755b10c5 (DEFAULT_TABLE table=test-table [id=76d57bde052d441592224773d14737ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:10.459240 15616 catalog_manager.cc:4841] TS b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889): tablet 32cd177bb89c4e4f80c5ac14042850ac (table test-table [id=94e9cb5b9f8944aaae466522d37238fe]) successfully deleted
I20260810 07:15:10.459481 15840 tablet_service.cc:1467] Processing CreateTablet for tablet d4243b45f7144cea8396e842755b10c5 (DEFAULT_TABLE table=test-table [id=76d57bde052d441592224773d14737ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:10.459549 16060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4243b45f7144cea8396e842755b10c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:10.459653 16285 tablet_service.cc:1467] Processing CreateTablet for tablet d4243b45f7144cea8396e842755b10c5 (DEFAULT_TABLE table=test-table [id=76d57bde052d441592224773d14737ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:10.459725 15840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4243b45f7144cea8396e842755b10c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:10.459873 16285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4243b45f7144cea8396e842755b10c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:10.461419 16632 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad: Bootstrap starting.
I20260810 07:15:10.461766 15615 catalog_manager.cc:4841] TS 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429): tablet 32cd177bb89c4e4f80c5ac14042850ac (table test-table [id=94e9cb5b9f8944aaae466522d37238fe]) successfully deleted
I20260810 07:15:10.462360 16633 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Bootstrap starting.
I20260810 07:15:10.462399 16632 tablet_bootstrap.cc:654] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:10.463307 16633 tablet_bootstrap.cc:654] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:10.463733 16632 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad: No bootstrap required, opened a new log
I20260810 07:15:10.463816 16632 ts_tablet_manager.cc:1397] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:10.464190 16634 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Bootstrap starting.
I20260810 07:15:10.464198 16632 raft_consensus.cc:348] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.464304 16632 raft_consensus.cc:374] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:10.464336 16632 raft_consensus.cc:729] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5cbc53ecf1248b6b133cb0fd089b4ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:10.464413 16633 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: No bootstrap required, opened a new log
I20260810 07:15:10.464481 16633 ts_tablet_manager.cc:1397] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 07:15:10.464476 16632 consensus_queue.cc:260] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.464934 16634 tablet_bootstrap.cc:654] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:10.464900 16633 raft_consensus.cc:348] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.464996 16633 raft_consensus.cc:374] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:10.465027 16633 raft_consensus.cc:729] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Initialized, Role: FOLLOWER
I20260810 07:15:10.465137 16633 consensus_queue.cc:260] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.465380 16633 ts_tablet_manager.cc:1428] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:10.466037 16632 ts_tablet_manager.cc:1428] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:10.466799 16634 tablet_bootstrap.cc:492] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: No bootstrap required, opened a new log
I20260810 07:15:10.466881 16634 ts_tablet_manager.cc:1397] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:10.467231 16634 raft_consensus.cc:348] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.467355 16634 raft_consensus.cc:374] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:10.467389 16634 raft_consensus.cc:729] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0df336b3d0145aaadf637bbbdcba656, State: Initialized, Role: FOLLOWER
I20260810 07:15:10.467497 16634 consensus_queue.cc:260] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.467737 16634 ts_tablet_manager.cc:1428] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:10.705068 16595 raft_consensus.cc:482] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:10.705197 16595 raft_consensus.cc:504] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.705559 16595 leader_election.cc:288] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:10.705971 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4243b45f7144cea8396e842755b10c5" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" is_pre_election: true
I20260810 07:15:10.706043 16310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4243b45f7144cea8396e842755b10c5" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656" is_pre_election: true
I20260810 07:15:10.706151 15868 raft_consensus.cc:2454] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 0.
I20260810 07:15:10.706207 16310 raft_consensus.cc:2454] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 0.
I20260810 07:15:10.706430 15989 leader_election.cc:302] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, f5cbc53ecf1248b6b133cb0fd089b4ad; no voters:
I20260810 07:15:10.706591 16595 raft_consensus.cc:2781] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:10.706646 16595 raft_consensus.cc:482] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:10.706689 16595 raft_consensus.cc:3037] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.707823 16595 raft_consensus.cc:504] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.708139 16595 leader_election.cc:288] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 election: Requested vote from peers f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919), b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889)
I20260810 07:15:10.708459 15868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4243b45f7144cea8396e842755b10c5" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad"
I20260810 07:15:10.708503 16310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4243b45f7144cea8396e842755b10c5" candidate_uuid: "9f130ee7d2ae402f992256590408d94d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0df336b3d0145aaadf637bbbdcba656"
I20260810 07:15:10.708595 16310 raft_consensus.cc:3037] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.708606 15868 raft_consensus.cc:3037] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:10.709483 16310 raft_consensus.cc:2454] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 1.
I20260810 07:15:10.709486 15868 raft_consensus.cc:2454] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f130ee7d2ae402f992256590408d94d in term 1.
I20260810 07:15:10.709731 15989 leader_election.cc:302] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f130ee7d2ae402f992256590408d94d, b0df336b3d0145aaadf637bbbdcba656; no voters:
I20260810 07:15:10.709895 16595 raft_consensus.cc:2781] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:10.710006 16595 raft_consensus.cc:686] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 1 LEADER]: Becoming Leader. State: Replica: 9f130ee7d2ae402f992256590408d94d, State: Running, Role: LEADER
I20260810 07:15:10.710140 16595 consensus_queue.cc:237] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } }
I20260810 07:15:10.711649 15637 catalog_manager.cc:5495] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f130ee7d2ae402f992256590408d94d (127.12.117.130). New cstate: current_term: 1 leader_uuid: "9f130ee7d2ae402f992256590408d94d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f130ee7d2ae402f992256590408d94d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 43429 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:10.764019 16310 raft_consensus.cc:1261] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Refusing update from remote peer 9f130ee7d2ae402f992256590408d94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:10.764108 15868 raft_consensus.cc:1261] T d4243b45f7144cea8396e842755b10c5 P f5cbc53ecf1248b6b133cb0fd089b4ad [term 1 FOLLOWER]: Refusing update from remote peer 9f130ee7d2ae402f992256590408d94d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:10.764380 16618 consensus_queue.cc:1035] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0df336b3d0145aaadf637bbbdcba656" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:10.764526 16595 consensus_queue.cc:1035] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5cbc53ecf1248b6b133cb0fd089b4ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:15:10.776484 16599 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 07:15:12.322472 15636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:12.322623 15636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:12.324765 15636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d81f75aafbc489da85b04bb5ae2d7d6: Sending DeleteTablet for 3 replicas of tablet d4243b45f7144cea8396e842755b10c5
I20260810 07:15:12.325295 16285 tablet_service.cc:1514] Processing DeleteTablet for tablet d4243b45f7144cea8396e842755b10c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:12 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:40042
I20260810 07:15:12.325294 15840 tablet_service.cc:1514] Processing DeleteTablet for tablet d4243b45f7144cea8396e842755b10c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:12 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:48900
I20260810 07:15:12.325402 16060 tablet_service.cc:1514] Processing DeleteTablet for tablet d4243b45f7144cea8396e842755b10c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:12 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59476
I20260810 07:15:12.325629 16687 tablet_replica.cc:330] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: stopping tablet replica
I20260810 07:15:12.325700 16691 tablet_replica.cc:330] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: stopping tablet replica
I20260810 07:15:12.325760 16687 raft_consensus.cc:2229] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:12.325846 16691 raft_consensus.cc:2229] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:12.325862 16687 raft_consensus.cc:2258] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:12.325992 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15752
I20260810 07:15:12.333302 16691 raft_consensus.cc:2258] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:12.333357 16687 ts_tablet_manager.cc:1905] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:12.333848 16691 ts_tablet_manager.cc:1905] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:12.336091 16687 ts_tablet_manager.cc:1918] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:15:12.336180 16687 log.cc:1196] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/d4243b45f7144cea8396e842755b10c5
I20260810 07:15:12.336329 16691 ts_tablet_manager.cc:1918] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 07:15:12.336416 16691 log.cc:1196] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/d4243b45f7144cea8396e842755b10c5
I20260810 07:15:12.336422 16687 ts_tablet_manager.cc:1939] T d4243b45f7144cea8396e842755b10c5 P b0df336b3d0145aaadf637bbbdcba656: Deleting consensus metadata
I20260810 07:15:12.336678 16691 ts_tablet_manager.cc:1939] T d4243b45f7144cea8396e842755b10c5 P 9f130ee7d2ae402f992256590408d94d: Deleting consensus metadata
I20260810 07:15:12.337088 15616 catalog_manager.cc:4841] TS b0df336b3d0145aaadf637bbbdcba656 (127.12.117.131:40889): tablet d4243b45f7144cea8396e842755b10c5 (table test-table [id=76d57bde052d441592224773d14737ce]) successfully deleted
I20260810 07:15:12.337386 15615 catalog_manager.cc:4841] TS 9f130ee7d2ae402f992256590408d94d (127.12.117.130:43429): tablet d4243b45f7144cea8396e842755b10c5 (table test-table [id=76d57bde052d441592224773d14737ce]) successfully deleted
W20260810 07:15:12.341019 15616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:42919 (error 108)
I20260810 07:15:12.341781 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15962
I20260810 07:15:12.342097 16694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:12.341690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42919 (local address 127.0.0.1:34510)
0810 07:15:12.341932 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:12.342007 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
W20260810 07:15:12.343786 15616 catalog_manager.cc:4584] TS f5cbc53ecf1248b6b133cb0fd089b4ad (127.12.117.129:42919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4243b45f7144cea8396e842755b10c5: Network error: Client connection negotiation failed: client connection to 127.12.117.129:42919: connect: Connection refused (error 111)
I20260810 07:15:12.352550 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16182
I20260810 07:15:12.363514 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15584
2026-08-10T07:15:12Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7007 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:12 dist-test-slave-gjrz krb5kdc[16698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:12 dist-test-slave-gjrz krb5kdc[16698](info): set up 2 sockets
Aug 10 07:15:12 dist-test-slave-gjrz krb5kdc[16698](info): commencing operation
krb5kdc: starting...
W20260810 07:15:13.524910 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:13 dist-test-slave-gjrz krb5kdc[16698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:13.636085 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34367
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:34367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:13.742954 16722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:13.743201 16722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:13.743257 16722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:13.747138 16722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:13.747189 16722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:13.747215 16722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:13.747236 16722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:13.747255 16722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:13.752297 16722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:34367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:13.753188 16722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:13.754107 16722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:13.762419 16736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:13.762521 16722 server_base.cc:1034] running on GCE node
W20260810 07:15:13.762431 16731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:13.762532 16733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:13.763329 16722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:13.764299 16722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:13.765455 16722 hybrid_clock.cc:648] HybridClock initialized: now 1786346113765440 us; error 28 us; skew 500 ppm
Aug 10 07:15:13 dist-test-slave-gjrz krb5kdc[16698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:13.767985 16722 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:13.768725 16722 webserver.cc:460] Webserver started at http://127.12.117.190:40279/ using document root <none> and password file <none>
I20260810 07:15:13.769183 16722 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:13.769235 16722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:13.769389 16722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:13.771253 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2"
format_stamp: "Formatted at 2026-08-10 07:15:13 on dist-test-slave-gjrz"
server_key: "235d2bd829c1484a80df69c1ca5da46b"
server_key_iv: "8a8654abda8d84f5524bcc01a6bf24f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:13.771780 16722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2"
format_stamp: "Formatted at 2026-08-10 07:15:13 on dist-test-slave-gjrz"
server_key: "235d2bd829c1484a80df69c1ca5da46b"
server_key_iv: "8a8654abda8d84f5524bcc01a6bf24f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:13.775017 16722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 07:15:13.777212 16748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:13.778290 16722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:13.778396 16722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2"
format_stamp: "Formatted at 2026-08-10 07:15:13 on dist-test-slave-gjrz"
server_key: "235d2bd829c1484a80df69c1ca5da46b"
server_key_iv: "8a8654abda8d84f5524bcc01a6bf24f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:13.778491 16722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:13.804131 16722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:13.806958 16722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:13.807119 16722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:13.813377 16722 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:34367
I20260810 07:15:13.813421 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:34367 every 8 connection(s)
I20260810 07:15:13.814229 16722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:13.816849 16832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:13.821133 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16722
I20260810 07:15:13.821219 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:13.821457 12758 external_mini_cluster.cc:1428] Setting key 097701f203eb6260aaf543ebe0778e41
I20260810 07:15:13.822219 16832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: Bootstrap starting.
I20260810 07:15:13.824772 16832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:13.825623 16832 log.cc:824] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:13.827589 16832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: No bootstrap required, opened a new log
Aug 10 07:15:13 dist-test-slave-gjrz krb5kdc[16698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:13.831334 16832 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } }
I20260810 07:15:13.831614 16832 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:13.831676 16832 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a0eaf50b3a4181a61ce3aeb9ef6bb2, State: Initialized, Role: FOLLOWER
I20260810 07:15:13.832163 16832 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } }
I20260810 07:15:13.832324 16832 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:13.832389 16832 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:13.832481 16832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:13.833425 16832 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } }
I20260810 07:15:13.833779 16832 leader_election.cc:302] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0a0eaf50b3a4181a61ce3aeb9ef6bb2; no voters:
I20260810 07:15:13.834118 16832 leader_election.cc:288] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:13.834175 16840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:13.834390 16840 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [term 1 LEADER]: Becoming Leader. State: Replica: b0a0eaf50b3a4181a61ce3aeb9ef6bb2, State: Running, Role: LEADER
I20260810 07:15:13.834738 16840 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } }
I20260810 07:15:13.835408 16832 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:13.836226 16843 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0a0eaf50b3a4181a61ce3aeb9ef6bb2. Latest consensus state: current_term: 1 leader_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } } }
I20260810 07:15:13.836350 16843 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:13.836411 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:13.822738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55858 (local address 127.12.117.190:34367)
0810 07:15:13.823147 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:13.823168 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:13.823214 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:15:13.824022 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:13.824046 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:13.824454 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:13.824812 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:13.825464 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.826760 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.827682 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:13.828354 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:13.828510 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:13.830722 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:13.830729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:13.830746 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:13.830778 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:13.832763 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.833208 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.833214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.833219 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.833368 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:13.833708 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:13.833712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:13.833713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:13.834227 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:13.834412 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:13.835151 (+ 739us) server_negotiation.cc:299] Negotiation successful
0810 07:15:13.835469 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":101,"threads_started":1}
I20260810 07:15:13.836676 16841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0a0eaf50b3a4181a61ce3aeb9ef6bb2" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34367 } } }
I20260810 07:15:13.836758 16841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:13.840211 16868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:13.840278 16868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:13.840335 16858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:13.841009 16858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:13.845964 16858 catalog_manager.cc:1347] Generated new cluster ID: 3847803dfcb54dd4992607d858a944ca
I20260810 07:15:13.846016 16858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:13.851737 16858 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:13.852559 16858 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:13.859239 16858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0a0eaf50b3a4181a61ce3aeb9ef6bb2: Generated new TSK 0
I20260810 07:15:13.859817 16858 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:13.889436 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:13.998858 16882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:13.999085 16882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:13.999125 16882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:14.003046 16882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:14.003099 16882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:14.003153 16882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:14.008126 16882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:14.008955 16882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:14.009812 16882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:15:14.018873 16882 server_base.cc:1034] running on GCE node
W20260810 07:15:14.018824 16895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:14.018833 16894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:14.018824 16897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:14.019477 16882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:14.020046 16882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:14.021206 16882 hybrid_clock.cc:648] HybridClock initialized: now 1786346114021189 us; error 28 us; skew 500 ppm
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:14.023752 16882 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:14.024536 16882 webserver.cc:460] Webserver started at http://127.12.117.129:41579/ using document root <none> and password file <none>
I20260810 07:15:14.025040 16882 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:14.025117 16882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:14.025318 16882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:14.027215 16882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "02098e4797da4f3cbccb6100a430e1f9"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "d6dd9e095a7648b77039b1372e1cfd5e"
server_key_iv: "1a989050682a74540b91c9246fd295a3"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.027783 16882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "02098e4797da4f3cbccb6100a430e1f9"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "d6dd9e095a7648b77039b1372e1cfd5e"
server_key_iv: "1a989050682a74540b91c9246fd295a3"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.031246 16882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:14.033568 16911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.034677 16882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:15:14.034783 16882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "02098e4797da4f3cbccb6100a430e1f9"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "d6dd9e095a7648b77039b1372e1cfd5e"
server_key_iv: "1a989050682a74540b91c9246fd295a3"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.034878 16882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:14.056542 16882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:14.059331 16882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:14.059501 16882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:14.060029 16882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:14.060812 16882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:14.060861 16882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.060911 16882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:14.060940 16882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.069312 16882 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:43183
I20260810 07:15:14.069355 17086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:43183 every 8 connection(s)
I20260810 07:15:14.070142 16882 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:14.075076 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16882
I20260810 07:15:14.075168 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:14.075488 12758 external_mini_cluster.cc:1428] Setting key fcf7b423705c629d5a139b1d0436d774
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:14.083804 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.071837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:40185 (local address 127.12.117.190:34367)
0810 07:15:14.071990 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.071995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.072839 (+ 844us) server_negotiation.cc:407] Connection header received
0810 07:15:14.074042 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.074045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.074094 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.074169 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:14.075323 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.076057 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.077363 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.077694 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.077797 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.080243 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:14.080246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:14.080249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:14.080276 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:14.081750 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.082287 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.082290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.082292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.082391 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.082690 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.082693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.082695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.082930 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:14.083031 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.083501 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.083633 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:14.084441 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.072271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34367 (local address 127.12.117.129:40185)
0810 07:15:14.072695 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.072736 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.073845 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.074310 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.074316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.074655 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:14.075126 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.075141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.076177 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.076181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.077234 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.077245 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.077826 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.077829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.077965 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.078511 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:14.078526 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:14.080074 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:14.081855 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.081860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.081881 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.082195 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.082472 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.082474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.082475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.082600 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.083033 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:14.083038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:14.083246 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.083435 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.083670 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":111,"threads_started":1}
I20260810 07:15:14.085502 17088 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34367
I20260810 07:15:14.085716 17088 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:14.086194 17088 heartbeater.cc:511] Master 127.12.117.190:34367 requested a full tablet report, sending...
I20260810 07:15:14.087620 16773 ts_manager.cc:194] Registered new tserver with Master: 02098e4797da4f3cbccb6100a430e1f9 (127.12.117.129:43183)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:15:14.088829 16773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:40185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:14.101673 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:14.210448 17094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:14.210675 17094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:14.210713 17094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:14.214677 17094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:14.214732 17094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:14.214789 17094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:14.219873 17094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:14.220757 17094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:14.221648 17094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:14.230602 17107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:14.230600 17101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:14.230711 17094 server_base.cc:1034] running on GCE node
W20260810 07:15:14.230598 17103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:14.231248 17094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:14.231858 17094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:14.233019 17094 hybrid_clock.cc:648] HybridClock initialized: now 1786346114233006 us; error 25 us; skew 500 ppm
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:14.235531 17094 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:14.236447 17094 webserver.cc:460] Webserver started at http://127.12.117.130:46473/ using document root <none> and password file <none>
I20260810 07:15:14.237004 17094 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:14.237092 17094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:14.237289 17094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:14.239334 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "358586f607b8499f9c455e83d0cd240f"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "3af9b71bbb82f599be493187eda006af"
server_key_iv: "e723dc522da8c4297765caaf01ecf4f4"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.239864 17094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "358586f607b8499f9c455e83d0cd240f"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "3af9b71bbb82f599be493187eda006af"
server_key_iv: "e723dc522da8c4297765caaf01ecf4f4"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.243352 17094 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:15:14.245599 17119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.246701 17094 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:14.246819 17094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "358586f607b8499f9c455e83d0cd240f"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "3af9b71bbb82f599be493187eda006af"
server_key_iv: "e723dc522da8c4297765caaf01ecf4f4"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.246912 17094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:14.266719 17094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:14.269271 17094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:14.269438 17094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:14.269969 17094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:14.270728 17094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:14.270777 17094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.270825 17094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:14.270855 17094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.279248 17094 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:35765
I20260810 07:15:14.279310 17291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:35765 every 8 connection(s)
I20260810 07:15:14.280076 17094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:14.286948 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17094
I20260810 07:15:14.287072 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:14.287415 12758 external_mini_cluster.cc:1428] Setting key 10d39d3191a8dfb394631badc78a2c85
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:14.293018 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.281698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:46573 (local address 127.12.117.190:34367)
0810 07:15:14.281847 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.281851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.282675 (+ 824us) server_negotiation.cc:407] Connection header received
0810 07:15:14.283957 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.283959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.284009 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.284074 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:14.285114 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.285635 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.286705 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.287013 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.287127 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.289595 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:14.289599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:14.289602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:14.289631 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:14.291017 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.291604 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.291607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.291608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.291686 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.292032 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.292034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.292035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.292200 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:14.292270 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.292801 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.292901 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:15:14.294037 17295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.282081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34367 (local address 127.12.117.130:46573)
0810 07:15:14.282516 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.282571 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.283754 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.284214 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.284220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.284559 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:14.285002 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.285016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.285728 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.285730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.286614 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.286623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.287166 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.287171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.287363 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.287948 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:14.287969 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:14.289420 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:14.291133 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.291139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.291153 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.291513 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.291762 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.291766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.291767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.291951 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.292295 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:14.292301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:14.292577 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.292768 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.293077 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":126,"threads_started":1}
I20260810 07:15:14.295143 17292 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34367
I20260810 07:15:14.295384 17292 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:14.295841 17292 heartbeater.cc:511] Master 127.12.117.190:34367 requested a full tablet report, sending...
I20260810 07:15:14.296828 16773 ts_manager.cc:194] Registered new tserver with Master: 358586f607b8499f9c455e83d0cd240f (127.12.117.130:35765)
I20260810 07:15:14.297600 16773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:46573
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:14.313274 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:14.420583 17302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:14.420797 17302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:14.420836 17302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:14.424722 17302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:14.424765 17302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:14.424818 17302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:14.429690 17302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:34367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:14.430496 17302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:14.431362 17302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:14.440090 17313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:14.440084 17309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:14.440285 17302 server_base.cc:1034] running on GCE node
W20260810 07:15:14.440084 17311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:14.440662 17302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:14.441200 17302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:14.442358 17302 hybrid_clock.cc:648] HybridClock initialized: now 1786346114442342 us; error 27 us; skew 500 ppm
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:14.444918 17302 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:14.445703 17302 webserver.cc:460] Webserver started at http://127.12.117.131:39177/ using document root <none> and password file <none>
I20260810 07:15:14.446172 17302 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:14.446228 17302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:14.446389 17302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:14.448256 17302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "66bda4dc864b41dc8a92e1b1b9339113"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "83263836e979ee72275e496efa2de528"
server_key_iv: "4ffec12adc4064e7dd90a5de04edb788"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.448748 17302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "66bda4dc864b41dc8a92e1b1b9339113"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "83263836e979ee72275e496efa2de528"
server_key_iv: "4ffec12adc4064e7dd90a5de04edb788"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.452250 17302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:15:14.454381 17325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.455494 17302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:14.455585 17302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "66bda4dc864b41dc8a92e1b1b9339113"
format_stamp: "Formatted at 2026-08-10 07:15:14 on dist-test-slave-gjrz"
server_key: "83263836e979ee72275e496efa2de528"
server_key_iv: "4ffec12adc4064e7dd90a5de04edb788"
server_key_version: "encryptionkey@0"
I20260810 07:15:14.455660 17302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:14.478345 17302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:14.481176 17302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:14.481338 17302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:14.481835 17302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:14.482602 17302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:14.482651 17302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.482688 17302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:14.482708 17302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:14.490917 17302 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:36857
I20260810 07:15:14.490960 17505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:36857 every 8 connection(s)
I20260810 07:15:14.491741 17302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:14.498415 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17302
I20260810 07:15:14.498525 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:14.498777 12758 external_mini_cluster.cc:1428] Setting key a90c121cc353c4580d746344d007cf02
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[16698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346114, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:14.504294 16837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.493407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:43461 (local address 127.12.117.190:34367)
0810 07:15:14.493534 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.493538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.494293 (+ 755us) server_negotiation.cc:407] Connection header received
0810 07:15:14.495578 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.495582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.495639 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.495711 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:14.496829 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.497361 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.498512 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.498776 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.498869 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.501348 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:14.501351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:14.501353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:14.501376 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:14.502665 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.503070 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.503072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.503074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.503145 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:14.503440 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:14.503442 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:14.503443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:14.503613 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:14.503689 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:14.504081 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 07:15:14.504178 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:15:14.505017 17512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:14.493732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:34367 (local address 127.12.117.131:43461)
0810 07:15:14.494150 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:14.494192 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:14.495365 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:14.495889 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:14.495896 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:14.496233 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:14.496681 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.496694 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.497480 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.497483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.498383 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:14.498394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:14.498920 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:14.498924 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:14.499121 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:14.499712 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:14.499728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:14.501200 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:14.502757 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.502761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.502771 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.502998 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.503208 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:14.503210 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:14.503212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:14.503368 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:14.503687 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:14.503692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:14.503902 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 07:15:14.504039 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 07:15:14.504265 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260810 07:15:14.506007 17508 heartbeater.cc:347] Connected to a master server at 127.12.117.190:34367
I20260810 07:15:14.506210 17508 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:14.506636 17508 heartbeater.cc:511] Master 127.12.117.190:34367 requested a full tablet report, sending...
I20260810 07:15:14.507560 16773 ts_manager.cc:194] Registered new tserver with Master: 66bda4dc864b41dc8a92e1b1b9339113 (127.12.117.131:36857)
I20260810 07:15:14.508107 16773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:43461
I20260810 07:15:14.512259 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:15:14.525913 16837 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:14.524616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.12.117.190:34367)
0810 07:15:14.524725 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:14.524730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:14.524906 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:15:14.525026 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:14.525029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:14.525070 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:14.525134 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:14.525791 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55864: BlockingRecv error: recv got EOF from 127.0.0.1:55864 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:14.526980 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16882
I20260810 07:15:14.565280 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17094
I20260810 07:15:14.576930 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17302
I20260810 07:15:14.593516 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16722
2026-08-10T07:15:14Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2231 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[17526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[17526](info): set up 2 sockets
Aug 10 07:15:14 dist-test-slave-gjrz krb5kdc[17526](info): commencing operation
krb5kdc: starting...
W20260810 07:15:15.757093 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:15 dist-test-slave-gjrz krb5kdc[17526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:15.870188 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37085
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:36711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:15.976847 17546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:15.977073 17546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:15.977110 17546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:15.980952 17546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:15.980996 17546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:15.981014 17546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:15.981035 17546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:15.981050 17546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:15.985985 17546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:36711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:15.986814 17546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:15.987691 17546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:15.995517 17556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:15.995568 17554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:15.995585 17552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:15.995678 17546 server_base.cc:1034] running on GCE node
I20260810 07:15:15.996326 17546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:15.997298 17546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:15.998469 17546 hybrid_clock.cc:648] HybridClock initialized: now 1786346115998449 us; error 26 us; skew 500 ppm
Aug 10 07:15:16 dist-test-slave-gjrz krb5kdc[17526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346116, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:16.003827 17546 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:16.004629 17546 webserver.cc:460] Webserver started at http://127.12.117.190:43339/ using document root <none> and password file <none>
I20260810 07:15:16.005106 17546 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:16.005160 17546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:16.005322 17546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:16.007210 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "17dd11213aad46429ba9d7477d921794"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-gjrz"
server_key: "5e0ca35cfc549b8370f375b3a13df866"
server_key_iv: "88b7050faa212da0f6499c5518b8f805"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.007736 17546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "17dd11213aad46429ba9d7477d921794"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-gjrz"
server_key: "5e0ca35cfc549b8370f375b3a13df866"
server_key_iv: "88b7050faa212da0f6499c5518b8f805"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.011101 17546 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:16.013338 17571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:16.014487 17546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:16.014599 17546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "17dd11213aad46429ba9d7477d921794"
format_stamp: "Formatted at 2026-08-10 07:15:16 on dist-test-slave-gjrz"
server_key: "5e0ca35cfc549b8370f375b3a13df866"
server_key_iv: "88b7050faa212da0f6499c5518b8f805"
server_key_version: "encryptionkey@0"
I20260810 07:15:16.014688 17546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:16.035576 17546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:16.038120 17546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:16.038292 17546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:16.044903 17546 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37085
I20260810 07:15:16.045147 17675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37085 every 8 connection(s)
I20260810 07:15:16.045809 17546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:16.048468 17676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:16.053838 17676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: Bootstrap starting.
I20260810 07:15:16.055169 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17546
I20260810 07:15:16.055282 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:16.055531 12758 external_mini_cluster.cc:1428] Setting key 74268976d67eb1a95ad95f998b17d24c
I20260810 07:15:16.056354 17676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:16.057257 17676 log.cc:824] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:16.059298 17676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: No bootstrap required, opened a new log
I20260810 07:15:16.063257 17676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } }
I20260810 07:15:16.063491 17676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:16.063542 17676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17dd11213aad46429ba9d7477d921794, State: Initialized, Role: FOLLOWER
I20260810 07:15:16.063936 17676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } }
I20260810 07:15:16.064059 17676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 10 07:15:16 dist-test-slave-gjrz krb5kdc[17526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:16.064124 17676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:16.064222 17676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:16.065167 17676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } }
I20260810 07:15:16.065533 17676 leader_election.cc:302] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17dd11213aad46429ba9d7477d921794; no voters:
I20260810 07:15:16.065903 17676 leader_election.cc:288] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:16.066005 17686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:16.066215 17686 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [term 1 LEADER]: Becoming Leader. State: Replica: 17dd11213aad46429ba9d7477d921794, State: Running, Role: LEADER
I20260810 07:15:16.066517 17686 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } }
I20260810 07:15:16.067116 17676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:16.068121 17687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17dd11213aad46429ba9d7477d921794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } } }
I20260810 07:15:16.068251 17687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:16.068630 17688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17dd11213aad46429ba9d7477d921794. Latest consensus state: current_term: 1 leader_uuid: "17dd11213aad46429ba9d7477d921794" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17dd11213aad46429ba9d7477d921794" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37085 } } }
I20260810 07:15:16.068792 17688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:16.070209 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.056476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.12.117.190:37085)
0810 07:15:16.056925 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.056943 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.056985 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:15:16.057769 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.057788 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.058162 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.058501 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.059155 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.060649 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.061480 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.062219 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.062394 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.064693 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.064699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.064716 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:16.064745 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.066543 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.067026 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.067032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.067037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.067193 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:16.067504 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:16.067507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:16.067508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:16.068014 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.068234 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.069081 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.069387 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":130,"threads_started":1}
W20260810 07:15:16.072211 17710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:16.072294 17710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:16.072353 17700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:16.072777 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:16.073225 17700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:16.079075 17700 catalog_manager.cc:1347] Generated new cluster ID: 7d18b5d66868450cad2f7993a5bfbe6b
I20260810 07:15:16.079133 17700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:16.091879 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:16.086614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.12.117.190:37085)
0810 07:15:16.086705 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:16.086709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:16.086895 (+ 186us) server_negotiation.cc:407] Connection header received
0810 07:15:16.086991 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:16.086994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:16.087041 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:16.087123 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:16.087691 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.088510 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.089594 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:16.089944 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:16.090041 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:16.090889 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:16.090893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:16.090896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:16.091405 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:16.091438 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:16.091457 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:16.091536 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:16.091689 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:15:16.091748 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":23}
I20260810 07:15:16.092363 17700 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:16.093317 17700 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:16.098946 17700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17dd11213aad46429ba9d7477d921794: Generated new TSK 0
I20260810 07:15:16.099639 17700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:16.108799 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17546
2026-08-10T07:15:16Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1517 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:16 dist-test-slave-gjrz krb5kdc[17726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:16 dist-test-slave-gjrz krb5kdc[17726](info): set up 2 sockets
Aug 10 07:15:16 dist-test-slave-gjrz krb5kdc[17726](info): commencing operation
krb5kdc: starting...
W20260810 07:15:17.277921 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:17.388505 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37699
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:17.496973 17747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:17.497349 17747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:17.497453 17747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:17.502871 17747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:17.502978 17747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:17.503051 17747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:17.503106 17747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:17.503159 17747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:17.508603 17747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:17.509469 17747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:17.510336 17747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:17.518249 17758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:17.518260 17755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:17.518253 17756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:17.518441 17747 server_base.cc:1034] running on GCE node
I20260810 07:15:17.519059 17747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:17.520067 17747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:17.521217 17747 hybrid_clock.cc:648] HybridClock initialized: now 1786346117521199 us; error 38 us; skew 500 ppm
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:17.523872 17747 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:17.524650 17747 webserver.cc:460] Webserver started at http://127.12.117.190:46353/ using document root <none> and password file <none>
I20260810 07:15:17.525162 17747 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:17.525219 17747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:17.525382 17747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:17.527315 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "134bd3508a5b4b41b465c0458c715461"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "7cc2d3e9e5492e692e761250fe5a24ae"
server_key_iv: "7843b984354ac693298f4bb2d6f833b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.527822 17747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "134bd3508a5b4b41b465c0458c715461"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "7cc2d3e9e5492e692e761250fe5a24ae"
server_key_iv: "7843b984354ac693298f4bb2d6f833b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.531239 17747 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:17.533505 17772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.534633 17747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:17.534744 17747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "134bd3508a5b4b41b465c0458c715461"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "7cc2d3e9e5492e692e761250fe5a24ae"
server_key_iv: "7843b984354ac693298f4bb2d6f833b2"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.534832 17747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:17.547524 17747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:17.549918 17747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:17.550074 17747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:17.556241 17747 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37699
I20260810 07:15:17.556459 17866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37699 every 8 connection(s)
I20260810 07:15:17.557090 17747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:17.559636 17868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:17.563666 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17747
I20260810 07:15:17.563766 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:17.564009 12758 external_mini_cluster.cc:1428] Setting key 56e8f9c3cf630443045c387ad4700e84
I20260810 07:15:17.564920 17868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: Bootstrap starting.
I20260810 07:15:17.567466 17868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:17.568243 17868 log.cc:824] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:17.570086 17868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: No bootstrap required, opened a new log
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:17.574251 17868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } }
I20260810 07:15:17.574507 17868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:17.574571 17868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 134bd3508a5b4b41b465c0458c715461, State: Initialized, Role: FOLLOWER
I20260810 07:15:17.575031 17868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } }
I20260810 07:15:17.575194 17868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:17.575284 17868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:17.575387 17868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:17.576324 17868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } }
I20260810 07:15:17.576678 17868 leader_election.cc:302] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 134bd3508a5b4b41b465c0458c715461; no voters:
I20260810 07:15:17.577028 17868 leader_election.cc:288] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:17.577116 17877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:17.577368 17877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [term 1 LEADER]: Becoming Leader. State: Replica: 134bd3508a5b4b41b465c0458c715461, State: Running, Role: LEADER
I20260810 07:15:17.577728 17877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } }
I20260810 07:15:17.578153 17868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:17.578886 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.565203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.12.117.190:37699)
0810 07:15:17.565597 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.565615 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.565653 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:15:17.566443 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.566467 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.566852 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.567200 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.567959 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.569282 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.570169 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.570827 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.570982 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.573485 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:17.573492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:17.573509 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:17.573540 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:17.575410 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.575845 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.575851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.575857 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.576016 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.576329 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.576332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.576334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.576809 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:17.577018 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.577698 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.577982 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260810 07:15:17.579385 17879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 134bd3508a5b4b41b465c0458c715461. Latest consensus state: current_term: 1 leader_uuid: "134bd3508a5b4b41b465c0458c715461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } } }
I20260810 07:15:17.579408 17878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "134bd3508a5b4b41b465c0458c715461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "134bd3508a5b4b41b465c0458c715461" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37699 } } }
I20260810 07:15:17.579519 17879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:17.579519 17878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461 [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:17.582605 17900 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:17.582671 17900 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:17.582916 17891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:17.583643 17891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:17.588553 17891 catalog_manager.cc:1347] Generated new cluster ID: a9b1a4f6c1034490853415ec38d5edb0
I20260810 07:15:17.588614 17891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:17.601886 17891 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:17.602670 17891 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:17.611495 17891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 134bd3508a5b4b41b465c0458c715461: Generated new TSK 0
I20260810 07:15:17.612084 17891 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:17.641772 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:17.749243 17908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:17.749472 17908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:17.749512 17908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:17.753508 17908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:17.753561 17908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:17.753625 17908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:17.758728 17908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:17.759680 17908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:17.760596 17908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:17.769439 17916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:17.769640 17908 server_base.cc:1034] running on GCE node
W20260810 07:15:17.769476 17920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:17.769485 17914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:17.770131 17908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:17.770665 17908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:17.771807 17908 hybrid_clock.cc:648] HybridClock initialized: now 1786346117771782 us; error 37 us; skew 500 ppm
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:17.774286 17908 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:17.775045 17908 webserver.cc:460] Webserver started at http://127.12.117.129:44559/ using document root <none> and password file <none>
I20260810 07:15:17.775592 17908 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:17.775658 17908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:17.775847 17908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:17.777707 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "fcbe72c5856349f7989395929aa866dd"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "a758ec8e9e2628fc89364911fe419ae4"
server_key_iv: "d2cce2e9ec652fe59354d6ebf8619cd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.778252 17908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "fcbe72c5856349f7989395929aa866dd"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "a758ec8e9e2628fc89364911fe419ae4"
server_key_iv: "d2cce2e9ec652fe59354d6ebf8619cd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.781852 17908 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:17.784240 17930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.785333 17908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:17.785444 17908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "fcbe72c5856349f7989395929aa866dd"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "a758ec8e9e2628fc89364911fe419ae4"
server_key_iv: "d2cce2e9ec652fe59354d6ebf8619cd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.785532 17908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:17.798138 17908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:17.800594 17908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:17.800748 17908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:17.801259 17908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:17.801998 17908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:17.802045 17908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.802093 17908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:17.802122 17908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.810196 17908 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:45099
I20260810 07:15:17.810250 18129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:45099 every 8 connection(s)
I20260810 07:15:17.810958 17908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:17.817101 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17908
I20260810 07:15:17.817232 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:17.817503 12758 external_mini_cluster.cc:1428] Setting key 8d72c6a4b40c02d6a31c633bd46bb0ce
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:17.823884 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.812510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:58415 (local address 127.12.117.190:37699)
0810 07:15:17.812629 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:17.812632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:17.813512 (+ 880us) server_negotiation.cc:407] Connection header received
0810 07:15:17.814760 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:17.814763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:17.814821 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:17.814898 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:17.815996 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.816529 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.818015 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.818294 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.818394 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.820707 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:17.820711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:17.820713 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:17.820739 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:17.821944 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.822465 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.822467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.822468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.822533 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:17.822848 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:17.822850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:17.822851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:17.823005 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:17.823088 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:17.823668 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 07:15:17.823767 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:17.824867 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:17.812933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37699 (local address 127.12.117.129:58415)
0810 07:15:17.813370 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:17.813409 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:17.814542 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:17.815064 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:17.815071 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:17.815414 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:17.815848 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.815861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.816651 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.816654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.817893 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:17.817904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:17.818449 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:17.818452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:17.818619 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:17.819129 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:17.819145 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:17.820543 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:17.822063 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:17.822069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:17.822082 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:17.822384 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:17.822608 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:17.822611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:17.822613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:17.822774 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:17.823112 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:17.823119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:17.823432 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 07:15:17.823635 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 07:15:17.823946 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260810 07:15:17.826021 18130 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37699
I20260810 07:15:17.826234 18130 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:17.826654 18130 heartbeater.cc:511] Master 127.12.117.190:37699 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:17.828095 17804 ts_manager.cc:194] Registered new tserver with Master: fcbe72c5856349f7989395929aa866dd (127.12.117.129:45099)
I20260810 07:15:17.829181 17804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:58415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:17.846690 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:17.957304 18140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:17.957566 18140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:17.957623 18140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:17.962718 18140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:17.962788 18140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:17.962868 18140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:17.968623 18140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:17.969537 18140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:17.970454 18140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:17.979427 18147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:17.979415 18146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:17.979417 18152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:17.979592 18140 server_base.cc:1034] running on GCE node
I20260810 07:15:17.980031 18140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:17.980611 18140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:17.981854 18140 hybrid_clock.cc:648] HybridClock initialized: now 1786346117981834 us; error 31 us; skew 500 ppm
Aug 10 07:15:17 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:17.984593 18140 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:17.985412 18140 webserver.cc:460] Webserver started at http://127.12.117.130:40355/ using document root <none> and password file <none>
I20260810 07:15:17.985916 18140 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:17.985991 18140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:17.986171 18140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:17.988165 18140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "f4b046fae5344efeaba099f0450e1466"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "bd8723d1c2724a65c44cb032d67ffdab"
server_key_iv: "69fd7443112185d69b604ffff4a7a1d4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.988708 18140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "f4b046fae5344efeaba099f0450e1466"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "bd8723d1c2724a65c44cb032d67ffdab"
server_key_iv: "69fd7443112185d69b604ffff4a7a1d4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.992247 18140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:17.994443 18171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:17.995630 18140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:17.995734 18140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "f4b046fae5344efeaba099f0450e1466"
format_stamp: "Formatted at 2026-08-10 07:15:17 on dist-test-slave-gjrz"
server_key: "bd8723d1c2724a65c44cb032d67ffdab"
server_key_iv: "69fd7443112185d69b604ffff4a7a1d4"
server_key_version: "encryptionkey@0"
I20260810 07:15:17.995819 18140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:18.014705 18140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:18.017225 18140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:18.017396 18140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:18.017935 18140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:18.018704 18140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:18.018751 18140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:18.018802 18140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:18.018831 18140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:18.027410 18140 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:46055
I20260810 07:15:18.027464 18352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:46055 every 8 connection(s)
I20260810 07:15:18.028244 18140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:18.032263 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18140
I20260810 07:15:18.032395 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:18.032701 12758 external_mini_cluster.cc:1428] Setting key 97ad09fbe858604fee669a18fc55d781
Aug 10 07:15:18 dist-test-slave-gjrz krb5kdc[17726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346117, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:18.042609 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:18.029917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:40351 (local address 127.12.117.190:37699)
0810 07:15:18.030048 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:18.030052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:18.030861 (+ 809us) server_negotiation.cc:407] Connection header received
0810 07:15:18.032179 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:18.032182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:18.032228 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:18.032298 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:18.033710 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:18.034294 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:18.036479 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:18.036706 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:18.036797 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:18.039173 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:18.039177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:18.039180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:18.039208 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:18.040572 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:18.041153 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:18.041155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:18.041156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:18.041228 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:18.041571 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:18.041572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:18.041574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:18.041728 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:18.041796 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:18.042393 (+ 597us) server_negotiation.cc:299] Negotiation successful
0810 07:15:18.042490 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:15:18.043669 18356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:18.030271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37699 (local address 127.12.117.130:40351)
0810 07:15:18.030718 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:18.030759 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:18.031964 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:18.032483 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:18.032491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:18.033015 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:18.033547 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:18.033578 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:18.034417 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:18.034421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:18.036358 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:18.036369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:18.036839 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:18.036843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:18.036997 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:18.037512 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:18.037528 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:18.038963 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:18.040689 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:18.040695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:18.040710 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:18.041048 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:18.041304 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:18.041307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:18.041309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:18.041488 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:18.041823 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:18.041830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:18.042157 (+ 327us) client_negotiation.cc:769] Sending connection context
0810 07:15:18.042376 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 07:15:18.042701 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260810 07:15:18.044893 18353 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37699
I20260810 07:15:18.045110 18353 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:18.045550 18353 heartbeater.cc:511] Master 127.12.117.190:37699 requested a full tablet report, sending...
I20260810 07:15:18.046491 17804 ts_manager.cc:194] Registered new tserver with Master: f4b046fae5344efeaba099f0450e1466 (127.12.117.130:46055)
I20260810 07:15:18.047013 17804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:40351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:15:18.831642 18130 heartbeater.cc:503] Master 127.12.117.190:37699 was elected leader, sending a full tablet report...
I20260810 07:15:19.048954 18353 heartbeater.cc:503] Master 127.12.117.190:37699 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
W20260810 07:15:19.270761 12758 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.12.117.131: real 1.236s user 0.000s sys 0.004s
I20260810 07:15:19.271297 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:19.382763 18360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:19.382984 18360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:19.383033 18360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:19.386914 18360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:19.386968 18360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:19.387028 18360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:19.392091 18360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:19.393023 18360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:19.394042 18360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:19.403143 18374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:19.403183 18371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:19.403371 18360 server_base.cc:1034] running on GCE node
W20260810 07:15:19.403183 18370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:19.403764 18360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:19.404382 18360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:19.405555 18360 hybrid_clock.cc:648] HybridClock initialized: now 1786346119405540 us; error 24 us; skew 500 ppm
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346119, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:19.408371 18360 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:19.409238 18360 webserver.cc:460] Webserver started at http://127.12.117.131:35455/ using document root <none> and password file <none>
I20260810 07:15:19.409740 18360 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:19.409804 18360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:19.410006 18360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:19.411970 18360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "77f898320ba140f69e761050325687d0"
format_stamp: "Formatted at 2026-08-10 07:15:19 on dist-test-slave-gjrz"
server_key: "b92384e6a19224c5f5d77eafe2f1e6f7"
server_key_iv: "69ad480ded134aa061c7b15f51a028a1"
server_key_version: "encryptionkey@0"
I20260810 07:15:19.412484 18360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "77f898320ba140f69e761050325687d0"
format_stamp: "Formatted at 2026-08-10 07:15:19 on dist-test-slave-gjrz"
server_key: "b92384e6a19224c5f5d77eafe2f1e6f7"
server_key_iv: "69ad480ded134aa061c7b15f51a028a1"
server_key_version: "encryptionkey@0"
I20260810 07:15:19.415897 18360 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:19.418067 18390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:19.419217 18360 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:19.419350 18360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "77f898320ba140f69e761050325687d0"
format_stamp: "Formatted at 2026-08-10 07:15:19 on dist-test-slave-gjrz"
server_key: "b92384e6a19224c5f5d77eafe2f1e6f7"
server_key_iv: "69ad480ded134aa061c7b15f51a028a1"
server_key_version: "encryptionkey@0"
I20260810 07:15:19.419437 18360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:19.433563 18360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:19.436331 18360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:19.436491 18360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:19.436988 18360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:19.437763 18360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:19.437810 18360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:19.437858 18360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:19.437887 18360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:19.445960 18360 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:40727
I20260810 07:15:19.445986 18591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:40727 every 8 connection(s)
I20260810 07:15:19.446712 18360 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:19.447767 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18360
I20260810 07:15:19.447888 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:19.448190 12758 external_mini_cluster.cc:1428] Setting key 9309aecc8bb80eefdffd5485c8dbccdd
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[17726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346119, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:19.459888 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:19.449700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:44943 (local address 127.12.117.190:37699)
0810 07:15:19.449926 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:19.449930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:19.449945 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:19.451595 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:19.451597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:19.451644 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:19.451718 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:19.452759 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.453277 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.454324 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.454546 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.454612 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:19.456856 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:19.456860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:19.456862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:19.456886 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:19.458250 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:19.458656 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:19.458658 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:19.458659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:19.458732 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:19.458971 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:19.458973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:19.458974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:19.459150 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:19.459222 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:19.459659 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 07:15:19.459760 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260810 07:15:19.460558 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:19.448821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37699 (local address 127.12.117.131:44943)
0810 07:15:19.449403 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:19.449444 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:19.451402 (+ 1958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:19.451855 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:19.451862 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:19.452196 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:19.452650 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:19.452663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.453361 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.453363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:19.454233 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:19.454243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.454646 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.454648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:19.454777 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:19.455285 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:19.455307 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:19.456710 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:19.458337 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:19.458340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:19.458350 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:19.458586 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:19.458794 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:19.458796 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:19.458797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:19.458910 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:19.459221 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:19.459225 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:19.459458 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 07:15:19.459614 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 07:15:19.459835 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":115,"threads_started":1}
I20260810 07:15:19.461534 18593 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37699
I20260810 07:15:19.461756 18593 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:19.462232 18593 heartbeater.cc:511] Master 127.12.117.190:37699 requested a full tablet report, sending...
I20260810 07:15:19.463150 17803 ts_manager.cc:194] Registered new tserver with Master: 77f898320ba140f69e761050325687d0 (127.12.117.131:40727)
I20260810 07:15:19.463722 17803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:44943
I20260810 07:15:19.473263 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[17726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346119, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
W20260810 07:15:19.580631 12758 mini_kdc.cc:317] Time spent kinit for joe-interloper: real 0.107s user 0.004s sys 0.000s
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[17726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346119, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:19.591496 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:19.582425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.12.117.190:37699)
0810 07:15:19.582563 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:19.582568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:19.582746 (+ 178us) server_negotiation.cc:407] Connection header received
0810 07:15:19.582892 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:19.582895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:19.582941 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:19.583011 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:19.584458 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.585033 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.585877 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:19.586109 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:19.586200 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:19.588526 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:19.588530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:19.588532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:19.588563 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:19.589819 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:19.590253 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:19.590256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:19.590257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:19.590338 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:19.590592 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:19.590595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:19.590597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:19.590822 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:19.590948 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:19.591193 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 07:15:19.591344 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 07:15:19.591881 17803 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38662
I20260810 07:15:19.592723 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17908
I20260810 07:15:19.606314 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18140
I20260810 07:15:19.621690 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18360
I20260810 07:15:19.631645 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17747
2026-08-10T07:15:19Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3516 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[18620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[18620](info): set up 2 sockets
Aug 10 07:15:19 dist-test-slave-gjrz krb5kdc[18620](info): commencing operation
krb5kdc: starting...
W20260810 07:15:20.786779 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:20 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:20.899560 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42505
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:42505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:21.005087 18637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.005308 18637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.005347 18637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.009147 18637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:21.009191 18637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.009208 18637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.009222 18637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:21.009236 18637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:21.014204 18637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:42505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:21.015014 18637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.015880 18637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.023716 18650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.023756 18637 server_base.cc:1034] running on GCE node
W20260810 07:15:21.023662 18654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.023662 18651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.024434 18637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.025429 18637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.026571 18637 hybrid_clock.cc:648] HybridClock initialized: now 1786346121026552 us; error 34 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.031769 18637 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:21.032603 18637 webserver.cc:460] Webserver started at http://127.12.117.190:35047/ using document root <none> and password file <none>
I20260810 07:15:21.033113 18637 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.033178 18637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.033357 18637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:21.035240 18637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "432e12f8382ad68b09e0c71237bac8e8"
server_key_iv: "7e737c36acbc78ee52a1d92077e567f6"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.035804 18637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "432e12f8382ad68b09e0c71237bac8e8"
server_key_iv: "7e737c36acbc78ee52a1d92077e567f6"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.039198 18637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:15:21.041420 18666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.042541 18637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:21.042654 18637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "432e12f8382ad68b09e0c71237bac8e8"
server_key_iv: "7e737c36acbc78ee52a1d92077e567f6"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.042739 18637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.052959 18637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.055660 18637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.055820 18637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.062362 18637 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:42505
I20260810 07:15:21.062397 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:42505 every 8 connection(s)
I20260810 07:15:21.063205 18637 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:21.064607 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18637
I20260810 07:15:21.064734 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:21.065028 12758 external_mini_cluster.cc:1428] Setting key 690438d21200fca123caed381d90e2c2
I20260810 07:15:21.066363 18754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:21.072445 18754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc: Bootstrap starting.
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:21.074769 18754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:21.075500 18754 log.cc:824] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:21.077500 18754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc: No bootstrap required, opened a new log
I20260810 07:15:21.078269 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.066284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58938 (local address 127.12.117.190:42505)
0810 07:15:21.066689 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.066704 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.066739 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:15:21.067378 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.067395 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.067712 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.067972 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.068859 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.069820 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.070930 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.071411 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.071524 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.074129 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.074135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.074147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.074173 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.075722 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.076130 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.076136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.076142 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.076259 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.076560 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.076562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.076563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.076911 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.077048 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.077341 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.077592 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":130,"threads_started":1}
I20260810 07:15:21.081133 18754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } }
I20260810 07:15:21.081337 18754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:21.081413 18754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44c63e9dd8ca45d6bcc10d669ceea5fc, State: Initialized, Role: FOLLOWER
I20260810 07:15:21.081786 18754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } }
I20260810 07:15:21.081931 18754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:21.081995 18754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:21.082093 18754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:21.082876 18754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } }
I20260810 07:15:21.083187 18754 leader_election.cc:302] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44c63e9dd8ca45d6bcc10d669ceea5fc; no voters:
I20260810 07:15:21.083477 18754 leader_election.cc:288] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:21.083600 18762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:21.083808 18762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [term 1 LEADER]: Becoming Leader. State: Replica: 44c63e9dd8ca45d6bcc10d669ceea5fc, State: Running, Role: LEADER
I20260810 07:15:21.084297 18762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } }
I20260810 07:15:21.084648 18754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:21.085807 18765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44c63e9dd8ca45d6bcc10d669ceea5fc. Latest consensus state: current_term: 1 leader_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } } }
I20260810 07:15:21.085934 18765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:21.086135 18766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44c63e9dd8ca45d6bcc10d669ceea5fc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42505 } } }
I20260810 07:15:21.086236 18766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:21.086540 18777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:21.089200 18777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:21.094776 18777 catalog_manager.cc:1347] Generated new cluster ID: fa2a79e21e244df48467b9acc276d7b7
I20260810 07:15:21.094835 18777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:21.108206 18777 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:21.109373 18777 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:21.118723 18777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44c63e9dd8ca45d6bcc10d669ceea5fc: Generated new TSK 0
I20260810 07:15:21.119431 18777 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:21.152571 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:21.261821 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.262060 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.262105 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.265998 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.266047 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.266106 18793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:21.271030 18793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:21.271921 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.272815 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.281729 18798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.281706 18802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.281720 18793 server_base.cc:1034] running on GCE node
W20260810 07:15:21.281749 18806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.282305 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.282872 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.284035 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786346121284018 us; error 35 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.286654 18793 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:21.287485 18793 webserver.cc:460] Webserver started at http://127.12.117.129:40683/ using document root <none> and password file <none>
I20260810 07:15:21.288021 18793 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.288087 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.288276 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:21.290109 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "73e83e0df00f47f6ae19492647695210"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "2ade3a1bfcb5dd2339665fd89e7a1afa"
server_key_iv: "41a0606dd985d80c0aaadbcab06da1ed"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.290625 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "73e83e0df00f47f6ae19492647695210"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "2ade3a1bfcb5dd2339665fd89e7a1afa"
server_key_iv: "41a0606dd985d80c0aaadbcab06da1ed"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.293956 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:21.296135 18816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.297230 18793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:21.297331 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "73e83e0df00f47f6ae19492647695210"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "2ade3a1bfcb5dd2339665fd89e7a1afa"
server_key_iv: "41a0606dd985d80c0aaadbcab06da1ed"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.297420 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.328410 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.330857 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.331033 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.331609 18793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.332376 18793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.332424 18793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.332474 18793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.332504 18793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.340423 18793 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:37893
I20260810 07:15:21.340476 18999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:37893 every 8 connection(s)
I20260810 07:15:21.341220 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:21.348698 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18793
I20260810 07:15:21.348829 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:21.349138 12758 external_mini_cluster.cc:1428] Setting key 00f41031d69ff709134c75f2b45030d0
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:21.354391 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.342858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:54175 (local address 127.12.117.190:42505)
0810 07:15:21.342978 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.342982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.343803 (+ 821us) server_negotiation.cc:407] Connection header received
0810 07:15:21.345153 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.345155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.345203 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.345273 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.346429 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.346975 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.348091 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.348313 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.348388 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.350931 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.350935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.350938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.350966 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.352378 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.352957 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.352960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.352961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.353039 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.353386 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.353388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.353389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.353554 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.353643 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.354176 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.354274 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:21.355367 19004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.343183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42505 (local address 127.12.117.129:54175)
0810 07:15:21.343654 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.343694 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.344955 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.345453 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.345460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.345800 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.346299 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.346314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.347101 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.347104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.347991 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.348001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.348423 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.348425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.348613 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.349173 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.349195 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.350767 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:21.352514 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.352520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.352535 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.352868 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.353120 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.353122 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.353124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.353303 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.353647 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:21.353653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:21.353939 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 07:15:21.354126 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 07:15:21.354423 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":119,"threads_started":1}
I20260810 07:15:21.356482 19001 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42505
I20260810 07:15:21.356683 19001 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:21.357106 19001 heartbeater.cc:511] Master 127.12.117.190:42505 requested a full tablet report, sending...
I20260810 07:15:21.358637 18692 ts_manager.cc:194] Registered new tserver with Master: 73e83e0df00f47f6ae19492647695210 (127.12.117.129:37893)
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:21.360141 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:54175
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:21.376073 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:21.482568 19008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.482784 19008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.482822 19008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.486938 19008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.486994 19008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.487047 19008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:21.492671 19008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:21.493839 19008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.495019 19008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.505232 19016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.505249 19017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.505368 19008 server_base.cc:1034] running on GCE node
W20260810 07:15:21.505237 19020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.505801 19008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.506361 19008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.507513 19008 hybrid_clock.cc:648] HybridClock initialized: now 1786346121507496 us; error 25 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.510067 19008 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:21.510876 19008 webserver.cc:460] Webserver started at http://127.12.117.130:46243/ using document root <none> and password file <none>
I20260810 07:15:21.511575 19008 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.511646 19008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.511816 19008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:21.513832 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "208ef8e21f3b4e78b755de2daa664d31"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "0bef5c30ae311dac7e3b43685235a000"
server_key_iv: "ec136bbf48912ee03b99052b67e6b077"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.514345 19008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "208ef8e21f3b4e78b755de2daa664d31"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "0bef5c30ae311dac7e3b43685235a000"
server_key_iv: "ec136bbf48912ee03b99052b67e6b077"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.517755 19008 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 07:15:21.520022 19030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.521159 19008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:21.521261 19008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "208ef8e21f3b4e78b755de2daa664d31"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "0bef5c30ae311dac7e3b43685235a000"
server_key_iv: "ec136bbf48912ee03b99052b67e6b077"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.521349 19008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.535948 19008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.538733 19008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.538888 19008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.539426 19008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.540169 19008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.540225 19008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.540275 19008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.540300 19008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.548646 19008 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:34717
I20260810 07:15:21.548668 19210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:34717 every 8 connection(s)
I20260810 07:15:21.549414 19008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:21.551491 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19008
I20260810 07:15:21.551620 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:21.551931 12758 external_mini_cluster.cc:1428] Setting key 21c5761a841b378654116942781f8a2a
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:15:21.563067 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.551043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:41209 (local address 127.12.117.190:42505)
0810 07:15:21.551162 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.551165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.552056 (+ 891us) server_negotiation.cc:407] Connection header received
0810 07:15:21.553498 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.553501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.553556 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.553628 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.554776 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.555470 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.556988 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.557205 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.557273 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.559844 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.559848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.559851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.559876 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.561115 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.561581 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.561585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.561587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.561668 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.561993 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.561997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.561998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.562187 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.562276 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.562778 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.562903 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:21.563760 19213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.551457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42505 (local address 127.12.117.130:41209)
0810 07:15:21.551916 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.551964 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.553307 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.553781 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.553788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.554098 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.554616 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.554631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.555595 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.555598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.556871 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.556883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.557337 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.557341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.557523 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.558143 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.558164 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.559690 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:21.561224 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.561230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.561244 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.561485 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.561753 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.561756 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.561759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.561903 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.562288 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:21.562294 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:21.562509 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 07:15:21.562675 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 07:15:21.562968 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:15:21.564883 19211 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42505
I20260810 07:15:21.565093 19211 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:21.565527 19211 heartbeater.cc:511] Master 127.12.117.190:42505 requested a full tablet report, sending...
I20260810 07:15:21.566486 18692 ts_manager.cc:194] Registered new tserver with Master: 208ef8e21f3b4e78b755de2daa664d31 (127.12.117.130:34717)
I20260810 07:15:21.567052 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:41209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:21.579478 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:21.685691 19216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:21.685913 19216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:21.685971 19216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:21.689888 19216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:21.689939 19216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:21.690016 19216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:21.694893 19216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:21.695744 19216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:21.696612 19216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:21.705513 19228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.705621 19216 server_base.cc:1034] running on GCE node
W20260810 07:15:21.705585 19225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:21.705513 19224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:21.706199 19216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:21.706774 19216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:21.707938 19216 hybrid_clock.cc:648] HybridClock initialized: now 1786346121707921 us; error 26 us; skew 500 ppm
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:21.710389 19216 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:21.711175 19216 webserver.cc:460] Webserver started at http://127.12.117.131:45521/ using document root <none> and password file <none>
I20260810 07:15:21.711699 19216 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:21.711762 19216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:21.711944 19216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:21.713845 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "fb833a5d775d4f2cb1ed252983100521"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "20d17e968f5a57a40ba647f52a6aa61f"
server_key_iv: "ebe1bc6e511c144d5f68fd1c3294ebe8"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.714355 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "fb833a5d775d4f2cb1ed252983100521"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "20d17e968f5a57a40ba647f52a6aa61f"
server_key_iv: "ebe1bc6e511c144d5f68fd1c3294ebe8"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.717751 19216 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:15:21.719929 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.720981 19216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:21.721087 19216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "fb833a5d775d4f2cb1ed252983100521"
format_stamp: "Formatted at 2026-08-10 07:15:21 on dist-test-slave-gjrz"
server_key: "20d17e968f5a57a40ba647f52a6aa61f"
server_key_iv: "ebe1bc6e511c144d5f68fd1c3294ebe8"
server_key_version: "encryptionkey@0"
I20260810 07:15:21.721174 19216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:21.733157 19216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:21.735580 19216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:21.735731 19216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:21.736224 19216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:21.736944 19216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:21.736991 19216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.737030 19216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:21.737066 19216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:21.744913 19216 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:39769
I20260810 07:15:21.744956 19445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:39769 every 8 connection(s)
I20260810 07:15:21.745707 19216 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:21.754642 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19216
I20260810 07:15:21.754760 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:21.755005 12758 external_mini_cluster.cc:1428] Setting key 0afb54bca5707d8e218c6ddf00408c35
I20260810 07:15:21.757958 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.747320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:43939 (local address 127.12.117.190:42505)
0810 07:15:21.747461 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.747465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.748249 (+ 784us) server_negotiation.cc:407] Connection header received
0810 07:15:21.749486 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.749489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.749535 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.749605 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.750624 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.751130 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.752250 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.752450 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.752511 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.754635 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.754638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.754640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.754664 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.756087 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.756630 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.756633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.756634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.756710 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.757036 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.757039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.757040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.757232 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.757310 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.757724 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.757817 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:21.758659 19451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.747688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42505 (local address 127.12.117.131:43939)
0810 07:15:21.748105 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:21.748145 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:21.749301 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:21.749739 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:21.749746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:21.750068 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:21.750513 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.750526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.751227 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.751230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.752155 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:21.752166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.752554 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.752556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:21.752691 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.753119 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:21.753134 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:21.754466 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:21.756210 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.756215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.756226 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.756535 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.756793 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:21.756797 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:21.756799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:21.756962 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:21.757320 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:21.757325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:21.757531 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 07:15:21.757691 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:15:21.757914 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
I20260810 07:15:21.759675 19447 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42505
I20260810 07:15:21.759884 19447 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:21.760324 19447 heartbeater.cc:511] Master 127.12.117.190:42505 requested a full tablet report, sending...
I20260810 07:15:21.761250 18692 ts_manager.cc:194] Registered new tserver with Master: fb833a5d775d4f2cb1ed252983100521 (127.12.117.131:39769)
I20260810 07:15:21.761772 18692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:43939
I20260810 07:15:21.768538 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:21.792680 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.784457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34926 (local address 127.12.117.129:37893)
0810 07:15:21.784811 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.784821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.784844 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:15:21.784887 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.784891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.785068 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.785187 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.785853 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.786608 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.787416 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.787759 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.787841 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.789881 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.789890 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.789901 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.789925 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.791187 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.791591 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.791596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.791597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.791674 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.791859 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.791861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.791862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.792075 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.792220 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.792377 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.792497 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":135,"threads_started":1}
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[18620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:21.802139 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:21.795158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.12.117.190:42505)
0810 07:15:21.795310 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:21.795314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:21.795347 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:15:21.795428 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:21.795430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:21.795466 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:21.795523 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:21.796029 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.796504 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.797194 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:21.797386 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:21.797446 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:21.799412 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:21.799415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:21.799417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:21.799440 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:21.800760 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.801125 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.801128 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.801129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.801205 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:21.801392 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:21.801394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:21.801395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:21.801621 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:21.801717 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:21.801903 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:15:21.801997 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 07:15:21.802488 18692 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58952
I20260810 07:15:21.803035 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18793
I20260810 07:15:21.828852 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19008
I20260810 07:15:21.840904 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19216
I20260810 07:15:21.851536 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18637
2026-08-10T07:15:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2221 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[19468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[19468](info): set up 2 sockets
Aug 10 07:15:21 dist-test-slave-gjrz krb5kdc[19468](info): commencing operation
krb5kdc: starting...
W20260810 07:15:23.004349 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:23.118275 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:33045
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:33045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:23.224116 19497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:23.224354 19497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:23.224411 19497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:23.228312 19497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:23.228363 19497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:23.228389 19497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:23.228410 19497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:23.228435 19497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:23.233417 19497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:33045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:33045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:23.234319 19497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:23.235219 19497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:23.242939 19512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.242949 19507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.242950 19508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:23.242983 19497 server_base.cc:1034] running on GCE node
I20260810 07:15:23.243726 19497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:23.244741 19497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:23.245915 19497 hybrid_clock.cc:648] HybridClock initialized: now 1786346123245892 us; error 37 us; skew 500 ppm
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:23.248539 19497 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:23.249224 19497 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:23.249293 19497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:23.249478 19497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:23.251385 19497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "eeb79de388c645ad9c3b4f720cb7be3a"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "3cab158758724a9463f18f609ddc4986"
server_key_iv: "73a321ba49e9d62468f3eeeca4f9e27e"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.251900 19497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "eeb79de388c645ad9c3b4f720cb7be3a"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "3cab158758724a9463f18f609ddc4986"
server_key_iv: "73a321ba49e9d62468f3eeeca4f9e27e"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.255195 19497 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:23.257345 19523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.258452 19497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:23.258563 19497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "eeb79de388c645ad9c3b4f720cb7be3a"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "3cab158758724a9463f18f609ddc4986"
server_key_iv: "73a321ba49e9d62468f3eeeca4f9e27e"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.258649 19497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:23.276500 19497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:23.279212 19497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:23.279393 19497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:23.285606 19497 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:33045
I20260810 07:15:23.285828 19608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:33045 every 8 connection(s)
I20260810 07:15:23.286175 19497 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:23.288781 19610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:23.293314 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19497
I20260810 07:15:23.293414 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:23.293655 12758 external_mini_cluster.cc:1428] Setting key 16813fad725860be49dba54ab7f663ac
I20260810 07:15:23.293953 19610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: Bootstrap starting.
I20260810 07:15:23.296161 19610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:23.296993 19610 log.cc:824] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:23.298938 19610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: No bootstrap required, opened a new log
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:23.303023 19610 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } }
I20260810 07:15:23.303241 19610 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:23.303316 19610 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb79de388c645ad9c3b4f720cb7be3a, State: Initialized, Role: FOLLOWER
I20260810 07:15:23.303808 19610 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } }
I20260810 07:15:23.303975 19610 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:23.304046 19610 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:23.304155 19610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:23.305128 19610 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } }
I20260810 07:15:23.305434 19610 leader_election.cc:302] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeb79de388c645ad9c3b4f720cb7be3a; no voters:
I20260810 07:15:23.305707 19610 leader_election.cc:288] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:23.305815 19617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:23.306036 19617 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [term 1 LEADER]: Becoming Leader. State: Replica: eeb79de388c645ad9c3b4f720cb7be3a, State: Running, Role: LEADER
I20260810 07:15:23.306363 19617 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } }
I20260810 07:15:23.306991 19610 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:23.307991 19621 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeb79de388c645ad9c3b4f720cb7be3a. Latest consensus state: current_term: 1 leader_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } } }
I20260810 07:15:23.308107 19621 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:23.308339 19619 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb79de388c645ad9c3b4f720cb7be3a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33045 } } }
I20260810 07:15:23.308418 19619 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:23.308418 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.294854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.12.117.190:33045)
0810 07:15:23.295210 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.295225 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.295258 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:15:23.295881 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.295896 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.296197 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.296474 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.297428 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.298525 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.299681 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.300165 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.300290 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.302944 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:23.302949 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:23.302961 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:23.302986 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:23.304922 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.305335 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.305341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.305346 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.305455 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.305748 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.305751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.305753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.306236 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:23.306991 (+ 755us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.307147 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.307471 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
I20260810 07:15:23.308481 19631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:23.311486 19631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:23.316162 19631 catalog_manager.cc:1347] Generated new cluster ID: 2e9edd461aeb4d6bbb5ae146eb19176f
I20260810 07:15:23.316216 19631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:23.324430 19631 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:23.325548 19631 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:23.330466 19631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: Generated new TSK 0
I20260810 07:15:23.331122 19631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:23.362283 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:23.468652 19655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:23.468864 19655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:23.468909 19655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:23.472752 19655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:23.472793 19655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:23.472862 19655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:23.477834 19655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:23.478684 19655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:23.479575 19655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:23.488157 19663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.488158 19669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.488158 19660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:23.488405 19655 server_base.cc:1034] running on GCE node
I20260810 07:15:23.488785 19655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:23.489326 19655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:23.490469 19655 hybrid_clock.cc:648] HybridClock initialized: now 1786346123490445 us; error 36 us; skew 500 ppm
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:23.493860 19655 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:23.494637 19655 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:23.494720 19655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:23.494983 19655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:23.497401 19655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "fb90dec85e2143aa9dc7479e430b90c8"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "0d4c1afa83f872d70d884e23806d1c89"
server_key_iv: "50991253a456deb0c521b7951765e127"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.497941 19655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "fb90dec85e2143aa9dc7479e430b90c8"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "0d4c1afa83f872d70d884e23806d1c89"
server_key_iv: "50991253a456deb0c521b7951765e127"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.501574 19655 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:23.503815 19680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.504922 19655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:23.505030 19655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "fb90dec85e2143aa9dc7479e430b90c8"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "0d4c1afa83f872d70d884e23806d1c89"
server_key_iv: "50991253a456deb0c521b7951765e127"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.505119 19655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:23.534878 19655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:23.537770 19655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:23.537935 19655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:23.538450 19655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:23.539218 19655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:23.539289 19655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.539351 19655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:23.539381 19655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.547387 19655 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:45397
I20260810 07:15:23.547447 19886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:45397 every 8 connection(s)
I20260810 07:15:23.547935 19655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:23.558109 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19655
I20260810 07:15:23.558226 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:23.558529 12758 external_mini_cluster.cc:1428] Setting key 276630d0a9d258fd27a26409aa4736a3
I20260810 07:15:23.560456 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.549488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:46535 (local address 127.12.117.190:33045)
0810 07:15:23.549620 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.549623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.550698 (+ 1075us) server_negotiation.cc:407] Connection header received
0810 07:15:23.551555 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.551558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.551614 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.551687 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.552798 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.553328 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.554442 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.554678 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.554769 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.556977 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:23.556980 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:23.556982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:23.557008 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:23.558115 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.558671 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.558674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.558675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.558754 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.559102 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.559107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.559108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.559429 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:23.559554 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.560235 (+ 681us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.560340 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:23.561220 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.549814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33045 (local address 127.12.117.129:46535)
0810 07:15:23.550559 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.550597 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.551347 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.551862 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.551869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.552204 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.552652 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.552665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.553452 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.553455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.554314 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.554324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.554815 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.554819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.554989 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.555462 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.555478 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.556803 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:23.558232 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.558239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.558253 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.558582 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.558839 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.558842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.558844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.558994 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.559571 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:23.559578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:23.559798 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 07:15:23.559977 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 07:15:23.560291 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":115,"threads_started":1}
I20260810 07:15:23.562400 19887 heartbeater.cc:347] Connected to a master server at 127.12.117.190:33045
I20260810 07:15:23.562644 19887 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:23.563055 19887 heartbeater.cc:511] Master 127.12.117.190:33045 requested a full tablet report, sending...
I20260810 07:15:23.564455 19553 ts_manager.cc:194] Registered new tserver with Master: fb90dec85e2143aa9dc7479e430b90c8 (127.12.117.129:45397)
I20260810 07:15:23.565560 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:46535
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:23.583712 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:23.690423 19894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:23.690634 19894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:23.690675 19894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:23.694556 19894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:23.694607 19894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:23.694664 19894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:23.699738 19894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:23.700606 19894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:23.701494 19894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:23.710384 19905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.710361 19909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.710403 19902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:23.710508 19894 server_base.cc:1034] running on GCE node
I20260810 07:15:23.710964 19894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:23.711596 19894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:23.712759 19894 hybrid_clock.cc:648] HybridClock initialized: now 1786346123712739 us; error 31 us; skew 500 ppm
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:23.715384 19894 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:23.716059 19894 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:23.716131 19894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:23.716316 19894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:23.718291 19894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "652afe82c2a04905b835ad4d11bd280d"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "6a3508ced8cec572642bdd079070d104"
server_key_iv: "9c8955922948eee8e684ef225fffe272"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.718815 19894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "652afe82c2a04905b835ad4d11bd280d"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "6a3508ced8cec572642bdd079070d104"
server_key_iv: "9c8955922948eee8e684ef225fffe272"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.722190 19894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:23.724433 19923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.725531 19894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:23.725637 19894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "652afe82c2a04905b835ad4d11bd280d"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "6a3508ced8cec572642bdd079070d104"
server_key_iv: "9c8955922948eee8e684ef225fffe272"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.725729 19894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:23.738417 19894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:23.740844 19894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:23.741000 19894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:23.741499 19894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:23.742244 19894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:23.742291 19894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.742341 19894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:23.742377 19894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.750520 19894 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:34731
I20260810 07:15:23.750571 20099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:34731 every 8 connection(s)
I20260810 07:15:23.751031 19894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:23.759202 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19894
I20260810 07:15:23.759327 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:23.759614 12758 external_mini_cluster.cc:1428] Setting key 401f22e4f2e4ef584e01f72dba5afb2e
I20260810 07:15:23.763245 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.752568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47125 (local address 127.12.117.190:33045)
0810 07:15:23.752705 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.752708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.753753 (+ 1045us) server_negotiation.cc:407] Connection header received
0810 07:15:23.754516 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.754519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.754563 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.754634 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.755853 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.756373 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.757502 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.757706 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.757786 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.760028 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:23.760031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:23.760033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:23.760056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:23.761330 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.761816 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.761819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.761821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.761918 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.762251 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.762254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.762256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.762426 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:23.762514 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.762982 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.763094 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:15:23.764070 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.752868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33045 (local address 127.12.117.130:47125)
0810 07:15:23.753616 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.753655 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.754348 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.754802 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.754808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.755130 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.755715 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.755730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.756472 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.756474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.757405 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.757415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.757813 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.757815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.757949 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.758423 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.758439 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.759872 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:23.761444 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.761450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.761464 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.761722 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.762025 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.762029 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.762031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.762164 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.762528 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:23.762534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:23.762751 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 07:15:23.762917 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 07:15:23.763225 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":112,"threads_started":1}
I20260810 07:15:23.765172 20100 heartbeater.cc:347] Connected to a master server at 127.12.117.190:33045
I20260810 07:15:23.765388 20100 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:23.765815 20100 heartbeater.cc:511] Master 127.12.117.190:33045 requested a full tablet report, sending...
I20260810 07:15:23.766723 19553 ts_manager.cc:194] Registered new tserver with Master: 652afe82c2a04905b835ad4d11bd280d (127.12.117.130:34731)
I20260810 07:15:23.767257 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:47125
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:23.784368 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:23.890424 20105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:23.890643 20105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:23.890682 20105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:23.894546 20105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:23.894594 20105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:23.894656 20105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:23.899668 20105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:33045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:23.900533 20105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:23.901417 20105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:23.910280 20114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.910254 20120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:23.910296 20115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:23.910346 20105 server_base.cc:1034] running on GCE node
I20260810 07:15:23.910852 20105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:23.911475 20105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:23.912631 20105 hybrid_clock.cc:648] HybridClock initialized: now 1786346123912615 us; error 26 us; skew 500 ppm
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:23.915200 20105 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:23.915895 20105 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:23.915966 20105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:23.916131 20105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:23.917991 20105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "13492a5c53ca43e1b16bed34e836b674"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "92b31db228c15c0b4f058c66d9c2252a"
server_key_iv: "32be6dc5b6f005cf2a2446708c0d7ee5"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.918485 20105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "13492a5c53ca43e1b16bed34e836b674"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "92b31db228c15c0b4f058c66d9c2252a"
server_key_iv: "32be6dc5b6f005cf2a2446708c0d7ee5"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.921829 20105 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:15:23.924079 20130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.925159 20105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:23.925251 20105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "13492a5c53ca43e1b16bed34e836b674"
format_stamp: "Formatted at 2026-08-10 07:15:23 on dist-test-slave-gjrz"
server_key: "92b31db228c15c0b4f058c66d9c2252a"
server_key_iv: "32be6dc5b6f005cf2a2446708c0d7ee5"
server_key_version: "encryptionkey@0"
I20260810 07:15:23.925329 20105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:23.937194 20105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:23.939630 20105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:23.939761 20105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:23.940217 20105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:23.940917 20105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:23.940964 20105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.941007 20105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:23.941028 20105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:23.949237 20105 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41385
I20260810 07:15:23.949266 20322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41385 every 8 connection(s)
I20260810 07:15:23.949736 20105 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:23 dist-test-slave-gjrz krb5kdc[19468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346123, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:23.959573 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 20105
I20260810 07:15:23.959656 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:23.959910 12758 external_mini_cluster.cc:1428] Setting key b899379802eb7621652fa64cf3e80f00
I20260810 07:15:23.961496 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.951298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:52481 (local address 127.12.117.190:33045)
0810 07:15:23.951424 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.951427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.952441 (+ 1014us) server_negotiation.cc:407] Connection header received
0810 07:15:23.953189 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.953192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.953236 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.953303 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.954290 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.954789 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.955850 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.956059 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.956127 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.958227 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:23.958231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:23.958233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:23.958256 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:23.959340 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.960083 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.960087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.960089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.960172 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.960494 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.960498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.960500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.960722 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:23.960811 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.961285 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.961385 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:23.962347 20331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.951581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33045 (local address 127.12.117.131:52481)
0810 07:15:23.952312 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.952348 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.953034 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.953435 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.953441 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.953763 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:23.954182 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.954195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.954879 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.954881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.955764 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.955774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.956159 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.956162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.956290 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.956775 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:23.956791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:23.958074 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:23.959715 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.959721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.959735 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.959988 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.960269 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:23.960272 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:23.960273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:23.960402 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:23.960817 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:23.960823 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:23.961078 (+ 255us) client_negotiation.cc:769] Sending connection context
0810 07:15:23.961246 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 07:15:23.961545 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":105,"threads_started":1}
I20260810 07:15:23.963356 20323 heartbeater.cc:347] Connected to a master server at 127.12.117.190:33045
I20260810 07:15:23.963541 20323 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:23.963891 20323 heartbeater.cc:511] Master 127.12.117.190:33045 requested a full tablet report, sending...
I20260810 07:15:23.964764 19553 ts_manager.cc:194] Registered new tserver with Master: 13492a5c53ca43e1b16bed34e836b674 (127.12.117.131:41385)
I20260810 07:15:23.965304 19553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:52481
I20260810 07:15:23.973343 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:23.981170 19614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.975008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.12.117.190:33045)
0810 07:15:23.975116 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.975119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.975343 (+ 224us) server_negotiation.cc:407] Connection header received
0810 07:15:23.975432 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.975435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.975475 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.975545 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:23.976139 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.976617 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.977351 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.977542 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.977605 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.978700 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:23.978703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:23.978705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:23.978727 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:23.979981 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.980299 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.980301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.980302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.980365 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:23.980539 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:23.980541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:23.980542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:23.980724 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:23.980819 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.980976 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.981063 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:23.983918 19553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53796:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:23.986680 19553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:23.999610 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.994228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45397 (local address 127.0.0.1:46670)
0810 07:15:23.994827 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.994862 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.995016 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.995645 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.995651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.995675 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:23.995793 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.995800 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.997051 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.997054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.997929 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.997941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.998991 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.998994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.999191 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.999240 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 07:15:23.999358 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 07:15:23.999478 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":67,"thread_start_us":74,"threads_started":1}
I20260810 07:15:23.999637 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.994359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.12.117.129:45397)
0810 07:15:23.994706 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.994718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.995159 (+ 441us) server_negotiation.cc:407] Connection header received
0810 07:15:23.995238 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.995243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.995481 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.995639 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:23.995934 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.996941 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.998068 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.998873 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.998986 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:23.999109 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:23.999392 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 07:15:23.999490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":131,"threads_started":1}
I20260810 07:15:24.000818 20348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.995045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41385 (local address 127.0.0.1:56614)
0810 07:15:23.996092 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.996102 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.996190 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.996590 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.996593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.996611 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:23.996706 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.996711 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.998005 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.998008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.999165 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.999175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.000305 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.000309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.000461 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.000514 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.000578 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.000651 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":127,"thread_start_us":73,"threads_started":1}
I20260810 07:15:24.000844 20349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.995428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.12.117.131:41385)
0810 07:15:23.995956 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.995967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.996186 (+ 219us) server_negotiation.cc:407] Connection header received
0810 07:15:23.996289 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.996293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.996455 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.996616 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:23.996792 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.997860 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.999321 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.000195 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.000306 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.000434 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.000620 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.000708 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":157,"threads_started":1}
I20260810 07:15:24.001096 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.994993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.12.117.130:34731)
0810 07:15:23.995330 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:23.995342 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:23.996000 (+ 658us) server_negotiation.cc:407] Connection header received
0810 07:15:23.996080 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:23.996085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:23.996261 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:23.996403 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:23.996644 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.997825 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.999307 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.000007 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.000123 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.000254 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.000888 (+ 634us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.000980 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260810 07:15:24.000820 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:23.994314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:34731 (local address 127.0.0.1:43430)
0810 07:15:23.995823 (+ 1509us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:23.995835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:23.995915 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:23.996395 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:23.996398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:23.996414 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:23.996518 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.996525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:23.997966 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:23.997969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:23.999172 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:23.999182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.000319 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.000323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.000486 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.000526 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.000595 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.000669 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1452,"mutex_wait_us":84,"thread_start_us":86,"threads_started":1}
I20260810 07:15:24.003082 19773 tablet_service.cc:1467] Processing CreateTablet for tablet 73895b18a6a9468586b199e3bb9418c1 (DEFAULT_TABLE table=test-table [id=3a3bd5d9f71644819d9cdf46e9b32549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:24.004242 19773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73895b18a6a9468586b199e3bb9418c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:24.004070 19992 tablet_service.cc:1467] Processing CreateTablet for tablet 73895b18a6a9468586b199e3bb9418c1 (DEFAULT_TABLE table=test-table [id=3a3bd5d9f71644819d9cdf46e9b32549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:24.004199 20210 tablet_service.cc:1467] Processing CreateTablet for tablet 73895b18a6a9468586b199e3bb9418c1 (DEFAULT_TABLE table=test-table [id=3a3bd5d9f71644819d9cdf46e9b32549]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:24.004941 19992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73895b18a6a9468586b199e3bb9418c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:24.005174 20210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73895b18a6a9468586b199e3bb9418c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:24.011734 20351 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Bootstrap starting.
I20260810 07:15:24.012130 20352 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Bootstrap starting.
I20260810 07:15:24.014040 20351 tablet_bootstrap.cc:654] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:24.014034 20352 tablet_bootstrap.cc:654] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:24.015206 20352 log.cc:824] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:24.015206 20351 log.cc:824] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:24.015583 20350 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Bootstrap starting.
I20260810 07:15:24.017282 20351 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: No bootstrap required, opened a new log
I20260810 07:15:24.017377 20352 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: No bootstrap required, opened a new log
I20260810 07:15:24.017467 20351 ts_tablet_manager.cc:1397] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:15:24.017562 20352 ts_tablet_manager.cc:1397] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:15:24.017686 20350 tablet_bootstrap.cc:654] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:24.018810 20350 log.cc:824] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:24.020522 20351 raft_consensus.cc:348] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.020702 20350 tablet_bootstrap.cc:492] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: No bootstrap required, opened a new log
I20260810 07:15:24.020722 20351 raft_consensus.cc:374] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:24.020802 20351 raft_consensus.cc:729] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb90dec85e2143aa9dc7479e430b90c8, State: Initialized, Role: FOLLOWER
I20260810 07:15:24.020867 20350 ts_tablet_manager.cc:1397] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:24.021217 20351 consensus_queue.cc:260] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.021184 20352 raft_consensus.cc:348] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.021476 20352 raft_consensus.cc:374] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:24.021544 20352 raft_consensus.cc:729] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13492a5c53ca43e1b16bed34e836b674, State: Initialized, Role: FOLLOWER
I20260810 07:15:24.021981 20352 consensus_queue.cc:260] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.022079 20351 ts_tablet_manager.cc:1428] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:15:24.022418 19887 heartbeater.cc:503] Master 127.12.117.190:33045 was elected leader, sending a full tablet report...
I20260810 07:15:24.022562 20323 heartbeater.cc:503] Master 127.12.117.190:33045 was elected leader, sending a full tablet report...
I20260810 07:15:24.022910 20352 ts_tablet_manager.cc:1428] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:15:24.025148 20350 raft_consensus.cc:348] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.025421 20350 raft_consensus.cc:374] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:24.025481 20350 raft_consensus.cc:729] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 652afe82c2a04905b835ad4d11bd280d, State: Initialized, Role: FOLLOWER
I20260810 07:15:24.025873 20350 consensus_queue.cc:260] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.026511 20100 heartbeater.cc:503] Master 127.12.117.190:33045 was elected leader, sending a full tablet report...
I20260810 07:15:24.026674 20350 ts_tablet_manager.cc:1428] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260810 07:15:24.048842 19888 tablet.cc:2369] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:24.175722 20360 raft_consensus.cc:482] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:24.175936 20360 raft_consensus.cc:504] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.177134 20360 leader_election.cc:288] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb90dec85e2143aa9dc7479e430b90c8 (127.12.117.129:45397), 652afe82c2a04905b835ad4d11bd280d (127.12.117.130:34731)
I20260810 07:15:24.180684 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.177476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:45679 (local address 127.12.117.129:45397)
0810 07:15:24.177585 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.177590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.177605 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:24.177658 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.177661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.177706 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.177771 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:24.178188 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.178688 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.179775 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.180172 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.180278 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.180333 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.180506 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.180578 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:24.180686 20331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.177301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45397 (local address 127.12.117.131:45679)
0810 07:15:24.177476 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.177490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.177586 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.177908 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.177911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.177953 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:24.178060 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.178064 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.178800 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.178802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.179644 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.179654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.180283 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.180287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.180389 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.180410 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.180477 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.180550 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":15,"spinlock_wait_cycles":31744}
I20260810 07:15:24.180820 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.177820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:38177 (local address 127.12.117.130:34731)
0810 07:15:24.177958 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.177962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.177976 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:24.178035 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.178038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.178079 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.178139 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:24.178378 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.178856 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.179912 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.180305 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.180408 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.180455 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.180637 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.180700 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 07:15:24.180845 20364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.177385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:34731 (local address 127.12.117.131:38177)
0810 07:15:24.177838 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.177850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.177958 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.178159 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.178161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.178176 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:24.178280 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.178285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.178955 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.178958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.179792 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:24.179802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.180416 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.180419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:24.180524 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.180544 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.180608 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.180675 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":16,"spinlock_wait_cycles":10880,"thread_start_us":118,"threads_started":1}
I20260810 07:15:24.181315 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73895b18a6a9468586b199e3bb9418c1" candidate_uuid: "13492a5c53ca43e1b16bed34e836b674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb90dec85e2143aa9dc7479e430b90c8" is_pre_election: true
I20260810 07:15:24.181391 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73895b18a6a9468586b199e3bb9418c1" candidate_uuid: "13492a5c53ca43e1b16bed34e836b674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652afe82c2a04905b835ad4d11bd280d" is_pre_election: true
I20260810 07:15:24.181643 20029 raft_consensus.cc:2454] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13492a5c53ca43e1b16bed34e836b674 in term 0.
I20260810 07:15:24.181643 19805 raft_consensus.cc:2454] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13492a5c53ca43e1b16bed34e836b674 in term 0.
I20260810 07:15:24.182144 20132 leader_election.cc:302] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13492a5c53ca43e1b16bed34e836b674, fb90dec85e2143aa9dc7479e430b90c8; no voters:
I20260810 07:15:24.182399 20360 raft_consensus.cc:2781] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:24.182483 20360 raft_consensus.cc:482] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:24.182544 20360 raft_consensus.cc:3037] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:24.183581 20360 raft_consensus.cc:504] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.183881 20360 leader_election.cc:288] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [CANDIDATE]: Term 1 election: Requested vote from peers fb90dec85e2143aa9dc7479e430b90c8 (127.12.117.129:45397), 652afe82c2a04905b835ad4d11bd280d (127.12.117.130:34731)
I20260810 07:15:24.184227 19805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73895b18a6a9468586b199e3bb9418c1" candidate_uuid: "13492a5c53ca43e1b16bed34e836b674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb90dec85e2143aa9dc7479e430b90c8"
I20260810 07:15:24.184260 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73895b18a6a9468586b199e3bb9418c1" candidate_uuid: "13492a5c53ca43e1b16bed34e836b674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "652afe82c2a04905b835ad4d11bd280d"
I20260810 07:15:24.184334 19805 raft_consensus.cc:3037] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:24.184355 20029 raft_consensus.cc:3037] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:24.185297 19805 raft_consensus.cc:2454] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13492a5c53ca43e1b16bed34e836b674 in term 1.
I20260810 07:15:24.185307 20029 raft_consensus.cc:2454] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13492a5c53ca43e1b16bed34e836b674 in term 1.
I20260810 07:15:24.185573 20135 leader_election.cc:302] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13492a5c53ca43e1b16bed34e836b674, 652afe82c2a04905b835ad4d11bd280d; no voters:
I20260810 07:15:24.185730 20360 raft_consensus.cc:2781] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:24.185940 20360 raft_consensus.cc:686] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 1 LEADER]: Becoming Leader. State: Replica: 13492a5c53ca43e1b16bed34e836b674, State: Running, Role: LEADER
I20260810 07:15:24.186251 20360 consensus_queue.cc:237] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } }
I20260810 07:15:24.189158 19553 catalog_manager.cc:5495] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13492a5c53ca43e1b16bed34e836b674 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "13492a5c53ca43e1b16bed34e836b674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13492a5c53ca43e1b16bed34e836b674" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41385 } health_report { overall_health: HEALTHY } } }
W20260810 07:15:24.200451 20324 tablet.cc:2369] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:24.230717 20349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.227361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.12.117.131:41385)
0810 07:15:24.227522 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.227526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.227541 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:24.227604 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.227606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.227654 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.227738 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:24.227966 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.228472 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.229430 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:24.229658 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:24.229735 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:24.230040 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:24.230421 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:24.230503 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.230545 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.230591 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 07:15:24.236495 19805 raft_consensus.cc:1261] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8 [term 1 FOLLOWER]: Refusing update from remote peer 13492a5c53ca43e1b16bed34e836b674: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:24.236569 20029 raft_consensus.cc:1261] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 1 FOLLOWER]: Refusing update from remote peer 13492a5c53ca43e1b16bed34e836b674: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:24.237171 20360 consensus_queue.cc:1035] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb90dec85e2143aa9dc7479e430b90c8" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:24.237339 20365 consensus_queue.cc:1035] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [LEADER]: Connected to new peer: Peer: permanent_uuid: "652afe82c2a04905b835ad4d11bd280d" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 34731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:24.252168 19553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:24.252305 19553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53796:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:24.254444 19553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eeb79de388c645ad9c3b4f720cb7be3a: Sending DeleteTablet for 3 replicas of tablet 73895b18a6a9468586b199e3bb9418c1
I20260810 07:15:24.255159 19773 tablet_service.cc:1514] Processing DeleteTablet for tablet 73895b18a6a9468586b199e3bb9418c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:24 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:46670
I20260810 07:15:24.255298 20210 tablet_service.cc:1514] Processing DeleteTablet for tablet 73895b18a6a9468586b199e3bb9418c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:24 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:56614
I20260810 07:15:24.255777 19992 tablet_service.cc:1514] Processing DeleteTablet for tablet 73895b18a6a9468586b199e3bb9418c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:24 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:43430
I20260810 07:15:24.255846 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19655
I20260810 07:15:24.255986 20386 tablet_replica.cc:330] T 73895b18a6a9468586b199e3bb9418c1 P fb90dec85e2143aa9dc7479e430b90c8: stopping tablet replica
I20260810 07:15:24.258935 20388 tablet_replica.cc:330] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: stopping tablet replica
I20260810 07:15:24.259196 20388 raft_consensus.cc:2229] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:24.259510 20388 raft_consensus.cc:2258] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:24.262987 20388 ts_tablet_manager.cc:1905] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:24.265405 20388 ts_tablet_manager.cc:1918] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:24.265502 20388 log.cc:1196] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/73895b18a6a9468586b199e3bb9418c1
I20260810 07:15:24.265796 20388 ts_tablet_manager.cc:1939] T 73895b18a6a9468586b199e3bb9418c1 P 13492a5c53ca43e1b16bed34e836b674: Deleting consensus metadata
I20260810 07:15:24.266721 19527 catalog_manager.cc:4841] TS 13492a5c53ca43e1b16bed34e836b674 (127.12.117.131:41385): tablet 73895b18a6a9468586b199e3bb9418c1 (table test-table [id=3a3bd5d9f71644819d9cdf46e9b32549]) successfully deleted
I20260810 07:15:24.267467 20391 tablet_replica.cc:330] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: stopping tablet replica
I20260810 07:15:24.267714 20391 raft_consensus.cc:2229] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:24.267982 20391 raft_consensus.cc:2258] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:24.269104 20391 ts_tablet_manager.cc:1905] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:15:24.271116 19526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:45397 (error 108)
I20260810 07:15:24.271140 20391 ts_tablet_manager.cc:1918] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:24.271224 20391 log.cc:1196] T 73895b18a6a9468586b199e3bb9418c1 P 652afe82c2a04905b835ad4d11bd280d: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/73895b18a6a9468586b199e3bb9418c1
I20260810 07:15:24.271301 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19894
I20260810 07:15:24.275875 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.273449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45397 (local address 127.0.0.1:46684)
0810 07:15:24.275734 (+ 2285us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.275802 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2198}
W20260810 07:15:24.294234 19526 catalog_manager.cc:4584] TS fb90dec85e2143aa9dc7479e430b90c8 (127.12.117.129:45397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73895b18a6a9468586b199e3bb9418c1: Network error: Client connection negotiation failed: client connection to 127.12.117.129:45397: connect: Connection refused (error 111)
I20260810 07:15:24.303294 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 20105
I20260810 07:15:24.306234 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.305972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:34731 (local address 127.0.0.1:43438)
0810 07:15:24.306105 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.306164 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.130:34731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 07:15:24.310518 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19497
2026-08-10T07:15:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2461 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T07:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:24Z Disabled control of system clock
I20260810 07:15:24.346181 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42233
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:24.453568 20399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:24.453787 20399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:24.453824 20399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:24.457705 20399 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:24.457753 20399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:24.457778 20399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:24.457808 20399 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:24.457834 20399 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:24.462416 20399 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:42233
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:24.463331 20399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:24.464591 20399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:24.472543 20409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:24.472687 20399 server_base.cc:1034] running on GCE node
W20260810 07:15:24.472525 20412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:24.472514 20408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:24.473368 20399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:24.474362 20399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:24.475517 20399 hybrid_clock.cc:648] HybridClock initialized: now 1786346124475500 us; error 25 us; skew 500 ppm
I20260810 07:15:24.477196 20399 webserver.cc:460] Webserver started at http://127.12.117.190:41107/ using document root <none> and password file <none>
I20260810 07:15:24.477713 20399 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:24.477773 20399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:24.477969 20399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:24.479943 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "876f9fe0e1ea44d5bbeaf4e248314131"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "c73c3fbfb9872fc50e96c8a8f299d331"
server_key_iv: "d9a6485618c0af4b378088f53343adad"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.480466 20399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "876f9fe0e1ea44d5bbeaf4e248314131"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "c73c3fbfb9872fc50e96c8a8f299d331"
server_key_iv: "d9a6485618c0af4b378088f53343adad"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.483937 20399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:24.486131 20422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.487299 20399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:24.487413 20399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "876f9fe0e1ea44d5bbeaf4e248314131"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "c73c3fbfb9872fc50e96c8a8f299d331"
server_key_iv: "d9a6485618c0af4b378088f53343adad"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.487502 20399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:24.491906 20399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:24.492457 20399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:24.492611 20399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:24.498764 20399 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:42233
I20260810 07:15:24.498808 20511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:42233 every 8 connection(s)
I20260810 07:15:24.499635 20399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:24.501106 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 20399
I20260810 07:15:24.501204 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:24.501442 12758 external_mini_cluster.cc:1428] Setting key ed16159593ad05ef24bce282d8b3f91b
I20260810 07:15:24.502859 20512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:24.506534 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.502723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.12.117.190:42233)
0810 07:15:24.503337 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.503353 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.503387 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:15:24.504004 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.504020 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.504338 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.504609 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.505392 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:24.505398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:24.505410 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:24.505420 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:24.505472 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:24.505487 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:24.505596 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.505681 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.505887 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":119,"threads_started":1}
I20260810 07:15:24.508504 20512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: Bootstrap starting.
I20260810 07:15:24.510469 20512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:24.511138 20512 log.cc:824] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:24.512825 20512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: No bootstrap required, opened a new log
I20260810 07:15:24.516044 20512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } }
I20260810 07:15:24.516244 20512 raft_consensus.cc:374] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:24.516302 20512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876f9fe0e1ea44d5bbeaf4e248314131, State: Initialized, Role: FOLLOWER
I20260810 07:15:24.516691 20512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } }
I20260810 07:15:24.516816 20512 raft_consensus.cc:388] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:24.516880 20512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:24.516983 20512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:24.517747 20512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } }
I20260810 07:15:24.518049 20512 leader_election.cc:302] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 876f9fe0e1ea44d5bbeaf4e248314131; no voters:
I20260810 07:15:24.518322 20512 leader_election.cc:288] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:24.518424 20519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:24.518597 20519 raft_consensus.cc:686] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [term 1 LEADER]: Becoming Leader. State: Replica: 876f9fe0e1ea44d5bbeaf4e248314131, State: Running, Role: LEADER
I20260810 07:15:24.519003 20519 consensus_queue.cc:237] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } }
I20260810 07:15:24.519516 20512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:24.520663 20522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 876f9fe0e1ea44d5bbeaf4e248314131. Latest consensus state: current_term: 1 leader_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } } }
I20260810 07:15:24.520678 20520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "876f9fe0e1ea44d5bbeaf4e248314131" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42233 } } }
I20260810 07:15:24.520798 20522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:24.520805 20520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:24.521185 20535 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:24.524168 20535 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:24.529558 20535 catalog_manager.cc:1347] Generated new cluster ID: db615c4cb81647a7bcd43cd6f8ad7549
I20260810 07:15:24.529608 20535 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:24.542416 20535 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:24.542739 20535 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:24.560588 20535 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: Generated new TSK 0
I20260810 07:15:24.561280 20535 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:24.572162 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--builtin_ntp_servers=127.12.117.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:24.683331 20554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:24.683547 20554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:24.683583 20554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:24.687427 20554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:24.687471 20554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:24.687521 20554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:24.692123 20554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:24.692988 20554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:24.694180 20554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:24.703424 20563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:24.703469 20562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:24.703423 20565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:24.703691 20554 server_base.cc:1034] running on GCE node
I20260810 07:15:24.704152 20554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:24.704730 20554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:24.705904 20554 hybrid_clock.cc:648] HybridClock initialized: now 1786346124705876 us; error 42 us; skew 500 ppm
I20260810 07:15:24.707691 20554 webserver.cc:460] Webserver started at http://127.12.117.129:42391/ using document root <none> and password file <none>
I20260810 07:15:24.708240 20554 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:24.708310 20554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:24.708519 20554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:24.710467 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "3dca163b517c4ffa991578bd052b5254"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "441eb671c1ffe79865616834fe048588"
server_key_iv: "8eb3054a69c3a6bf16e3e881d64f1f81"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.711002 20554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "3dca163b517c4ffa991578bd052b5254"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "441eb671c1ffe79865616834fe048588"
server_key_iv: "8eb3054a69c3a6bf16e3e881d64f1f81"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.714570 20554 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:24.716969 20575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.718093 20554 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:24.718204 20554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "3dca163b517c4ffa991578bd052b5254"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "441eb671c1ffe79865616834fe048588"
server_key_iv: "8eb3054a69c3a6bf16e3e881d64f1f81"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.718303 20554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:24.722994 20554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:24.723618 20554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:24.723770 20554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:24.724268 20554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:24.725129 20554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:24.725178 20554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.725227 20554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:24.725256 20554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.733309 20554 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:46781
I20260810 07:15:24.733337 20783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:46781 every 8 connection(s)
I20260810 07:15:24.734093 20554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:24.738514 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 20554
I20260810 07:15:24.738634 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:24.738929 12758 external_mini_cluster.cc:1428] Setting key 6e349c5bebd5cdb24f4b421ed42eafa2
I20260810 07:15:24.739907 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.735735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:42563 (local address 127.12.117.190:42233)
0810 07:15:24.735857 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.735861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.736835 (+ 974us) server_negotiation.cc:407] Connection header received
0810 07:15:24.738151 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.738154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.738190 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.738262 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.739484 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:24.739488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:24.739491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:24.739505 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:24.739526 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:24.739535 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:24.739604 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.739744 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.739783 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:24.740749 20787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.736108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42233 (local address 127.12.117.129:42563)
0810 07:15:24.736706 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.736742 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.737952 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.738454 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.738462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.738592 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:24.739162 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:24.739164 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:24.739211 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:24.739327 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:24.739609 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:24.739617 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:24.739624 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.739759 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.739931 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":99,"threads_started":1}
I20260810 07:15:24.740938 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--builtin_ntp_servers=127.12.117.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 07:15:24.741912 20784 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42233
I20260810 07:15:24.742118 20784 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:24.742496 20784 heartbeater.cc:511] Master 127.12.117.190:42233 requested a full tablet report, sending...
I20260810 07:15:24.743983 20458 ts_manager.cc:194] Registered new tserver with Master: 3dca163b517c4ffa991578bd052b5254 (127.12.117.129:46781)
W20260810 07:15:24.848589 20788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:24.848800 20788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:24.848852 20788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:24.852737 20788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:24.852782 20788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:24.852855 20788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:24.857508 20788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:24.858372 20788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:24.859588 20788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:24.868271 20797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:24.868280 20799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:24.868356 20788 server_base.cc:1034] running on GCE node
W20260810 07:15:24.868323 20803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:24.868863 20788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:24.869449 20788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:24.870631 20788 hybrid_clock.cc:648] HybridClock initialized: now 1786346124870609 us; error 33 us; skew 500 ppm
I20260810 07:15:24.872380 20788 webserver.cc:460] Webserver started at http://127.12.117.130:45253/ using document root <none> and password file <none>
I20260810 07:15:24.872900 20788 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:24.872967 20788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:24.873145 20788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:24.875031 20788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "243c38350e38467a99f71be536e7b2cf"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "26459795d459bc3bcf6ae2fe2ce4a73b"
server_key_iv: "79b04c69d6e4dd6b23e13c89024a517e"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.875582 20788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "243c38350e38467a99f71be536e7b2cf"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "26459795d459bc3bcf6ae2fe2ce4a73b"
server_key_iv: "79b04c69d6e4dd6b23e13c89024a517e"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.879117 20788 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:15:24.881444 20810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.882543 20788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:24.882664 20788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "243c38350e38467a99f71be536e7b2cf"
format_stamp: "Formatted at 2026-08-10 07:15:24 on dist-test-slave-gjrz"
server_key: "26459795d459bc3bcf6ae2fe2ce4a73b"
server_key_iv: "79b04c69d6e4dd6b23e13c89024a517e"
server_key_version: "encryptionkey@0"
I20260810 07:15:24.882763 20788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:24.887037 20788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:24.887629 20788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:24.887781 20788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:24.888275 20788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:24.889165 20788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:24.889219 20788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.889269 20788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:24.889298 20788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:24.897595 20788 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:38267
I20260810 07:15:24.897616 21009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:38267 every 8 connection(s)
I20260810 07:15:24.898404 20788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:24.903782 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.900069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:36091 (local address 127.12.117.190:42233)
0810 07:15:24.900211 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:24.900214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:24.901171 (+ 957us) server_negotiation.cc:407] Connection header received
0810 07:15:24.902364 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:24.902367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:24.902405 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:24.902472 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:24.903401 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:24.903404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:24.903407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:24.903419 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:24.903439 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:24.903447 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:24.903520 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:24.903642 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:15:24.903678 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:15:24.904551 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:24.900377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42233 (local address 127.12.117.130:36091)
0810 07:15:24.901036 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:24.901075 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:24.902195 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:24.902594 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:24.902600 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:24.902681 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:24.903122 (+ 441us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:24.903123 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:24.903159 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:24.903223 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:24.903532 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:24.903538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:24.903542 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:24.903669 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 07:15:24.903857 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":122,"threads_started":1}
I20260810 07:15:24.905578 21010 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42233
I20260810 07:15:24.905777 21010 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:24.906126 21010 heartbeater.cc:511] Master 127.12.117.190:42233 requested a full tablet report, sending...
I20260810 07:15:24.907063 20458 ts_manager.cc:194] Registered new tserver with Master: 243c38350e38467a99f71be536e7b2cf (127.12.117.130:38267)
I20260810 07:15:24.907861 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 20788
I20260810 07:15:24.907974 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:24.908223 12758 external_mini_cluster.cc:1428] Setting key 0c6fbdbffe739611e540c8d406ce8d11
I20260810 07:15:24.909983 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--builtin_ntp_servers=127.12.117.148:33511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 07:15:25.020042 21013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:25.020294 21013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:25.020334 21013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:25.024461 21013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:25.024515 21013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:25.024583 21013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:25.029299 21013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:25.030230 21013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:25.031522 21013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:25.040715 21024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:25.040694 21021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:25.040719 21026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:25.040894 21013 server_base.cc:1034] running on GCE node
I20260810 07:15:25.041280 21013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:25.041879 21013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:25.043040 21013 hybrid_clock.cc:648] HybridClock initialized: now 1786346125043022 us; error 33 us; skew 500 ppm
I20260810 07:15:25.044792 21013 webserver.cc:460] Webserver started at http://127.12.117.131:35863/ using document root <none> and password file <none>
I20260810 07:15:25.045338 21013 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:25.045406 21013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:25.045593 21013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:25.047580 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "650ca1d839a24f388eab4875461d9d08"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "ca1b8673ec99a824eb7f830ca783fbd8"
server_key_iv: "12651426ba806b9102efdd3d600dfe5e"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.048128 21013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "650ca1d839a24f388eab4875461d9d08"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "ca1b8673ec99a824eb7f830ca783fbd8"
server_key_iv: "12651426ba806b9102efdd3d600dfe5e"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.051573 21013 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:25.053916 21039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:25.055049 21013 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:25.055166 21013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "650ca1d839a24f388eab4875461d9d08"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "ca1b8673ec99a824eb7f830ca783fbd8"
server_key_iv: "12651426ba806b9102efdd3d600dfe5e"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.055281 21013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:25.059509 21013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:25.060075 21013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:25.060228 21013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:25.060781 21013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:25.061628 21013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:25.061674 21013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:25.061710 21013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:25.061731 21013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:25.069857 21013 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41463
I20260810 07:15:25.069880 21236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41463 every 8 connection(s)
I20260810 07:15:25.070641 21013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:25.076200 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.072285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35569 (local address 127.12.117.190:42233)
0810 07:15:25.072414 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.072418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.073389 (+ 971us) server_negotiation.cc:407] Connection header received
0810 07:15:25.074625 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.074628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.074675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.074747 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.075780 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.075784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.075787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.075799 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.075820 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.075828 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.075903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.076022 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.076067 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:15:25.076236 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 21013
I20260810 07:15:25.076341 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:25.076598 12758 external_mini_cluster.cc:1428] Setting key e031ac59c6b3820ec155a9268da9d1f2
I20260810 07:15:25.077064 21241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.072641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42233 (local address 127.12.117.131:35569)
0810 07:15:25.073254 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.073291 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.074423 (+ 1132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.074910 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.074917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.075000 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.075532 (+ 532us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.075534 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.075571 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.075637 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.075895 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.075903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.075909 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.076064 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.076292 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":107,"threads_started":1}
I20260810 07:15:25.078106 21238 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42233
I20260810 07:15:25.078327 21238 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:25.078722 21238 heartbeater.cc:511] Master 127.12.117.190:42233 requested a full tablet report, sending...
I20260810 07:15:25.079644 20458 ts_manager.cc:194] Registered new tserver with Master: 650ca1d839a24f388eab4875461d9d08 (127.12.117.131:41463)
I20260810 07:15:25.090176 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:25.093262 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.092000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54892 (local address 127.12.117.190:42233)
0810 07:15:25.092127 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.092131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.092260 (+ 129us) server_negotiation.cc:407] Connection header received
0810 07:15:25.092368 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.092370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.092400 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.092454 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.092970 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.092973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.092975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.092984 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.093000 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.093007 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.093081 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.093118 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.093146 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:25.095685 20458 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:25.098485 20458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:25.109175 21261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.106224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46781 (local address 127.0.0.1:38514)
0810 07:15:25.106841 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.106853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.106910 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.107938 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.107941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.107964 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.108450 (+ 486us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.108452 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.108471 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.108535 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.108873 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.108882 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.108887 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.108988 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.109055 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":78,"threads_started":1}
I20260810 07:15:25.109220 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.106854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38514 (local address 127.12.117.129:46781)
0810 07:15:25.107545 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.107557 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.107582 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:15:25.107633 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.107638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.107809 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.107962 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.108693 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.108700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.108712 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.108724 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.108764 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.108801 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.108891 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.109014 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.109093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":102,"threads_started":1}
I20260810 07:15:25.109318 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.106099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38267 (local address 127.0.0.1:58556)
0810 07:15:25.106580 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.106629 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.106723 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.107891 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.107897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.107964 (+ 67us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.108685 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.108687 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.108698 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.108723 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.109081 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.109084 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.109087 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.109159 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.109193 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":75,"threads_started":1}
I20260810 07:15:25.109848 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.106254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.12.117.130:38267)
0810 07:15:25.106674 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.106686 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.106710 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:15:25.106849 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.106856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.107051 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.107186 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.108871 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.108878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.108890 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.108905 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.108959 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.108992 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.109096 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.109630 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.109707 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":106,"threads_started":1}
I20260810 07:15:25.110119 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.107318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36004 (local address 127.12.117.131:41463)
0810 07:15:25.107702 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.107715 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.107738 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:15:25.107785 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.107790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.107946 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.108084 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.109523 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.109531 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.109542 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.109557 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.109599 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.109630 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.109727 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.109821 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.109888 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":133,"threads_started":1}
I20260810 07:15:25.110579 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.106779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41463 (local address 127.0.0.1:36004)
0810 07:15:25.107078 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.107092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.107168 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.108549 (+ 1381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.108552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.108566 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.109167 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.109169 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.109179 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.109202 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.109717 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.109720 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.109723 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.110429 (+ 706us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.110466 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":10,"thread_start_us":83,"threads_started":1}
I20260810 07:15:25.112210 20660 tablet_service.cc:1467] Processing CreateTablet for tablet c40333a28be1457d9fd02925fa0760f9 (DEFAULT_TABLE table=test-table [id=61dd8611248e4627aeef5e1aaae11fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:25.112399 20887 tablet_service.cc:1467] Processing CreateTablet for tablet c40333a28be1457d9fd02925fa0760f9 (DEFAULT_TABLE table=test-table [id=61dd8611248e4627aeef5e1aaae11fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:25.113334 20660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40333a28be1457d9fd02925fa0760f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:25.113405 20887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40333a28be1457d9fd02925fa0760f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:25.113343 21115 tablet_service.cc:1467] Processing CreateTablet for tablet c40333a28be1457d9fd02925fa0760f9 (DEFAULT_TABLE table=test-table [id=61dd8611248e4627aeef5e1aaae11fbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:25.114228 21115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c40333a28be1457d9fd02925fa0760f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:25.119295 21269 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Bootstrap starting.
I20260810 07:15:25.121021 21271 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Bootstrap starting.
I20260810 07:15:25.121021 21272 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Bootstrap starting.
I20260810 07:15:25.121888 21269 tablet_bootstrap.cc:654] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:25.122757 21269 log.cc:824] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:25.123106 21271 tablet_bootstrap.cc:654] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:25.123106 21272 tablet_bootstrap.cc:654] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:25.123936 21272 log.cc:824] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:25.123936 21271 log.cc:824] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:25.124505 21269 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: No bootstrap required, opened a new log
I20260810 07:15:25.124701 21269 ts_tablet_manager.cc:1397] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:15:25.125759 21271 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: No bootstrap required, opened a new log
I20260810 07:15:25.125759 21272 tablet_bootstrap.cc:492] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: No bootstrap required, opened a new log
I20260810 07:15:25.125937 21272 ts_tablet_manager.cc:1397] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:15:25.125937 21271 ts_tablet_manager.cc:1397] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:15:25.128202 21269 raft_consensus.cc:348] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.128402 21269 raft_consensus.cc:374] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:25.128463 21269 raft_consensus.cc:729] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243c38350e38467a99f71be536e7b2cf, State: Initialized, Role: FOLLOWER
I20260810 07:15:25.128850 21269 consensus_queue.cc:260] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.129449 21010 heartbeater.cc:503] Master 127.12.117.190:42233 was elected leader, sending a full tablet report...
I20260810 07:15:25.129735 21269 ts_tablet_manager.cc:1428] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:15:25.130172 21271 raft_consensus.cc:348] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.130414 21271 raft_consensus.cc:374] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:25.130468 21271 raft_consensus.cc:729] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dca163b517c4ffa991578bd052b5254, State: Initialized, Role: FOLLOWER
I20260810 07:15:25.130648 21272 raft_consensus.cc:348] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.130862 21272 raft_consensus.cc:374] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:25.130918 21272 raft_consensus.cc:729] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650ca1d839a24f388eab4875461d9d08, State: Initialized, Role: FOLLOWER
I20260810 07:15:25.130908 21271 consensus_queue.cc:260] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.131337 21272 consensus_queue.cc:260] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.131548 20784 heartbeater.cc:503] Master 127.12.117.190:42233 was elected leader, sending a full tablet report...
I20260810 07:15:25.131935 21238 heartbeater.cc:503] Master 127.12.117.190:42233 was elected leader, sending a full tablet report...
I20260810 07:15:25.132244 21272 ts_tablet_manager.cc:1428] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:15:25.132896 21271 ts_tablet_manager.cc:1428] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260810 07:15:25.149250 21011 tablet.cc:2369] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:25.235118 20786 tablet.cc:2369] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:25.283351 21281 raft_consensus.cc:482] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:25.283536 21281 raft_consensus.cc:504] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.284582 21281 leader_election.cc:288] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3dca163b517c4ffa991578bd052b5254 (127.12.117.129:46781), 650ca1d839a24f388eab4875461d9d08 (127.12.117.131:41463)
I20260810 07:15:25.286283 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.284912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47487 (local address 127.12.117.129:46781)
0810 07:15:25.285069 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.285073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.285088 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:25.285132 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.285134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.285166 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.285242 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.285933 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.285937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.285939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.285953 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.285972 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.285980 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.286063 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.286130 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.286169 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 07:15:25.286283 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.284828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46781 (local address 127.12.117.130:47487)
0810 07:15:25.284927 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.284941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.285021 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.285295 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.285298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.285316 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.285773 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.285774 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.285784 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.285810 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.286048 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.286050 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.286053 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.286122 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.286160 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260810 07:15:25.286764 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.284936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:44591 (local address 127.12.117.131:41463)
0810 07:15:25.285051 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.285056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.285474 (+ 418us) server_negotiation.cc:407] Connection header received
0810 07:15:25.285544 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.285546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.285582 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.285648 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.286239 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.286242 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.286245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.286257 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.286295 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.286305 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.286403 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.286603 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.286647 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:25.286761 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.284828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41463 (local address 127.12.117.130:44591)
0810 07:15:25.285373 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.285384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.285469 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.285683 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.285686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.285701 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:25.286092 (+ 391us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:25.286094 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:25.286104 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:25.286130 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:25.286481 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:25.286484 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:25.286488 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:15:25.286584 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:15:25.286620 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":247,"thread_start_us":121,"threads_started":1}
I20260810 07:15:25.286962 20696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40333a28be1457d9fd02925fa0760f9" candidate_uuid: "243c38350e38467a99f71be536e7b2cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dca163b517c4ffa991578bd052b5254" is_pre_election: true
I20260810 07:15:25.287315 20696 raft_consensus.cc:2454] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243c38350e38467a99f71be536e7b2cf in term 0.
I20260810 07:15:25.287508 21156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40333a28be1457d9fd02925fa0760f9" candidate_uuid: "243c38350e38467a99f71be536e7b2cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650ca1d839a24f388eab4875461d9d08" is_pre_election: true
I20260810 07:15:25.287786 21156 raft_consensus.cc:2454] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 243c38350e38467a99f71be536e7b2cf in term 0.
I20260810 07:15:25.287801 20812 leader_election.cc:302] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243c38350e38467a99f71be536e7b2cf, 3dca163b517c4ffa991578bd052b5254; no voters:
I20260810 07:15:25.288079 21281 raft_consensus.cc:2781] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:25.288179 21281 raft_consensus.cc:482] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:25.288230 21281 raft_consensus.cc:3037] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:25.289258 21281 raft_consensus.cc:504] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.289556 21281 leader_election.cc:288] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [CANDIDATE]: Term 1 election: Requested vote from peers 3dca163b517c4ffa991578bd052b5254 (127.12.117.129:46781), 650ca1d839a24f388eab4875461d9d08 (127.12.117.131:41463)
I20260810 07:15:25.289891 21156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40333a28be1457d9fd02925fa0760f9" candidate_uuid: "243c38350e38467a99f71be536e7b2cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650ca1d839a24f388eab4875461d9d08"
I20260810 07:15:25.289897 20696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c40333a28be1457d9fd02925fa0760f9" candidate_uuid: "243c38350e38467a99f71be536e7b2cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dca163b517c4ffa991578bd052b5254"
I20260810 07:15:25.290028 20696 raft_consensus.cc:3037] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:25.290020 21156 raft_consensus.cc:3037] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:25.291333 20696 raft_consensus.cc:2454] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243c38350e38467a99f71be536e7b2cf in term 1.
I20260810 07:15:25.291333 21156 raft_consensus.cc:2454] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 243c38350e38467a99f71be536e7b2cf in term 1.
I20260810 07:15:25.291653 20811 leader_election.cc:302] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 243c38350e38467a99f71be536e7b2cf, 650ca1d839a24f388eab4875461d9d08; no voters:
I20260810 07:15:25.291870 21281 raft_consensus.cc:2781] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:25.292079 21281 raft_consensus.cc:686] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 1 LEADER]: Becoming Leader. State: Replica: 243c38350e38467a99f71be536e7b2cf, State: Running, Role: LEADER
I20260810 07:15:25.292368 21281 consensus_queue.cc:237] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } }
I20260810 07:15:25.295204 20457 catalog_manager.cc:5495] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 243c38350e38467a99f71be536e7b2cf (127.12.117.130). New cstate: current_term: 1 leader_uuid: "243c38350e38467a99f71be536e7b2cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "243c38350e38467a99f71be536e7b2cf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 } health_report { overall_health: UNKNOWN } } }
W20260810 07:15:25.321451 21240 tablet.cc:2369] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:25.340005 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.338672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.12.117.130:38267)
0810 07:15:25.338775 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.338778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.338791 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:25.338919 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.338921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.338978 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.339044 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.339620 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.339623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.339625 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.339636 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.339654 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.339662 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.339733 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.339848 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.339882 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:15:25.345424 20696 raft_consensus.cc:1261] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 1 FOLLOWER]: Refusing update from remote peer 243c38350e38467a99f71be536e7b2cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:25.345425 21156 raft_consensus.cc:1261] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 1 FOLLOWER]: Refusing update from remote peer 243c38350e38467a99f71be536e7b2cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:25.346093 21286 consensus_queue.cc:1035] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "650ca1d839a24f388eab4875461d9d08" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:25.346258 21281 consensus_queue.cc:1035] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dca163b517c4ffa991578bd052b5254" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:25.353976 21290 mvcc.cc:205] Tried to move back new op lower bound from 7316873729408602112 to 7316873729199812608. Current Snapshot: MvccSnapshot[applied={T|T < 7316873729408602112}]
I20260810 07:15:25.361487 20457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:25.361611 20457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:25.363788 20457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 876f9fe0e1ea44d5bbeaf4e248314131: Sending DeleteTablet for 3 replicas of tablet c40333a28be1457d9fd02925fa0760f9
I20260810 07:15:25.364519 20660 tablet_service.cc:1514] Processing DeleteTablet for tablet c40333a28be1457d9fd02925fa0760f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:25 UTC) from {username='slave'} at 127.0.0.1:38514
I20260810 07:15:25.364544 21115 tablet_service.cc:1514] Processing DeleteTablet for tablet c40333a28be1457d9fd02925fa0760f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:25 UTC) from {username='slave'} at 127.0.0.1:36004
I20260810 07:15:25.364665 20887 tablet_service.cc:1514] Processing DeleteTablet for tablet c40333a28be1457d9fd02925fa0760f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:25 UTC) from {username='slave'} at 127.0.0.1:58556
I20260810 07:15:25.365031 21309 tablet_replica.cc:330] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: stopping tablet replica
I20260810 07:15:25.365028 21311 tablet_replica.cc:330] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: stopping tablet replica
I20260810 07:15:25.365307 21311 raft_consensus.cc:2229] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:25.365451 21310 tablet_replica.cc:330] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254: stopping tablet replica
I20260810 07:15:25.365654 21311 raft_consensus.cc:2258] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:25.365803 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 20554
I20260810 07:15:25.365799 21310 raft_consensus.cc:2229] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:25.366163 21310 raft_consensus.cc:2258] T c40333a28be1457d9fd02925fa0760f9 P 3dca163b517c4ffa991578bd052b5254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:25.367170 21311 ts_tablet_manager.cc:1905] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:25.369957 21311 ts_tablet_manager.cc:1918] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:25.370067 21311 log.cc:1196] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/c40333a28be1457d9fd02925fa0760f9
I20260810 07:15:25.370404 21311 ts_tablet_manager.cc:1939] T c40333a28be1457d9fd02925fa0760f9 P 243c38350e38467a99f71be536e7b2cf: Deleting consensus metadata
I20260810 07:15:25.371482 20423 catalog_manager.cc:4841] TS 243c38350e38467a99f71be536e7b2cf (127.12.117.130:38267): tablet c40333a28be1457d9fd02925fa0760f9 (table test-table [id=61dd8611248e4627aeef5e1aaae11fbb]) successfully deleted
I20260810 07:15:25.375554 21309 raft_consensus.cc:2229] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:25.375862 21309 raft_consensus.cc:2258] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:15:25.375909 20425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:46781 (error 108)
I20260810 07:15:25.376219 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 20788
I20260810 07:15:25.377058 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.376761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46781 (local address 127.0.0.1:38524)
0810 07:15:25.376903 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.376980 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 07:15:25.379024 21309 ts_tablet_manager.cc:1905] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:25.381278 21309 ts_tablet_manager.cc:1918] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 07:15:25.381280 20425 catalog_manager.cc:4584] TS 3dca163b517c4ffa991578bd052b5254 (127.12.117.129:46781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c40333a28be1457d9fd02925fa0760f9: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46781: connect: Connection refused (error 111)
I20260810 07:15:25.381364 21309 log.cc:1196] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/c40333a28be1457d9fd02925fa0760f9
I20260810 07:15:25.381635 21309 ts_tablet_manager.cc:1939] T c40333a28be1457d9fd02925fa0760f9 P 650ca1d839a24f388eab4875461d9d08: Deleting consensus metadata
I20260810 07:15:25.387534 20423 catalog_manager.cc:4841] TS 650ca1d839a24f388eab4875461d9d08 (127.12.117.131:41463): tablet c40333a28be1457d9fd02925fa0760f9 (table test-table [id=61dd8611248e4627aeef5e1aaae11fbb]) successfully deleted
I20260810 07:15:25.388322 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 21013
I20260810 07:15:25.402573 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 20399
2026-08-10T07:15:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1091 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 07:15:25.429935 12758 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:25.431524 12758 webserver.cc:460] Webserver started at https://127.0.0.1:45737/ using document root <none> and password file <none>
I20260810 07:15:25.431692 12758 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:25.431939 12758 webserver.cc:460] Webserver started at http://127.0.0.1:41905/ using document root <none> and password file <none>
2026-08-10T07:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:25Z Disabled control of system clock
I20260810 07:15:25.440789 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42951
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:38403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:42951
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45737/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/testchainca.cert with env {}
W20260810 07:15:25.548678 21320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:25.548899 21320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:25.548938 21320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:25.552807 21320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:25.552852 21320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:25.552865 21320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:25.552881 21320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:25.552893 21320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:25.552907 21320 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:25.552927 21320 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45737/jwks.json
I20260810 07:15:25.557922 21320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:38403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:42951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45737/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:25.558758 21320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:25.559650 21320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:25.567497 21329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:25.567514 21328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:25.567580 21320 server_base.cc:1034] running on GCE node
W20260810 07:15:25.567519 21331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:25.568171 21320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:25.569149 21320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:25.570312 21320 hybrid_clock.cc:648] HybridClock initialized: now 1786346125570296 us; error 27 us; skew 500 ppm
I20260810 07:15:25.572065 21320 webserver.cc:460] Webserver started at http://127.12.117.190:46153/ using document root <none> and password file <none>
I20260810 07:15:25.572582 21320 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:25.572649 21320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:25.572824 21320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:25.574677 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "86f71c52541c42aab579abdd8379e012"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "f9b766db7c086497677ab3a8f07ee00a"
server_key_iv: "ebe007883d0b8c11bf71b3deafb5ead6"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.575191 21320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "86f71c52541c42aab579abdd8379e012"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "f9b766db7c086497677ab3a8f07ee00a"
server_key_iv: "ebe007883d0b8c11bf71b3deafb5ead6"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.578512 21320 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:25.580708 21345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:25.581813 21320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:25.581945 21320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "86f71c52541c42aab579abdd8379e012"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "f9b766db7c086497677ab3a8f07ee00a"
server_key_iv: "ebe007883d0b8c11bf71b3deafb5ead6"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.582036 21320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:25.594827 21320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:25.595427 21320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:25.595582 21320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:25.602097 21320 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:42951
I20260810 07:15:25.602102 21437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:42951 every 8 connection(s)
I20260810 07:15:25.602916 21320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:25.605593 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 21320
I20260810 07:15:25.605695 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:25.605947 12758 external_mini_cluster.cc:1428] Setting key d39d4cf156224ebd4d509982da54ca20
I20260810 07:15:25.606050 21439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:25.612983 21439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012: Bootstrap starting.
I20260810 07:15:25.615447 21439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:25.615754 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.607279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53326 (local address 127.12.117.190:42951)
0810 07:15:25.607941 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.607961 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.608002 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:15:25.608801 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.608823 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.609302 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.609665 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.610080 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.611481 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.612322 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.612983 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.613140 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.613842 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.613848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.613863 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.614336 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.614402 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.614428 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.614518 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.614612 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.614854 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":103,"threads_started":1}
I20260810 07:15:25.616253 21439 log.cc:824] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:25.618139 21439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012: No bootstrap required, opened a new log
I20260810 07:15:25.618350 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:25.621654 21439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } }
I20260810 07:15:25.621862 21439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:25.621932 21439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f71c52541c42aab579abdd8379e012, State: Initialized, Role: FOLLOWER
I20260810 07:15:25.622385 21439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } }
I20260810 07:15:25.622535 21439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:25.622601 21439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:25.622694 21439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:25.623528 21439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } }
I20260810 07:15:25.623833 21439 leader_election.cc:302] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f71c52541c42aab579abdd8379e012; no voters:
I20260810 07:15:25.624111 21439 leader_election.cc:288] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:25.624192 21450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:25.624408 21450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [term 1 LEADER]: Becoming Leader. State: Replica: 86f71c52541c42aab579abdd8379e012, State: Running, Role: LEADER
I20260810 07:15:25.624688 21450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } }
I20260810 07:15:25.625223 21439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:25.626055 21452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f71c52541c42aab579abdd8379e012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } } }
I20260810 07:15:25.626179 21452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:25.626426 21453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f71c52541c42aab579abdd8379e012. Latest consensus state: current_term: 1 leader_uuid: "86f71c52541c42aab579abdd8379e012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f71c52541c42aab579abdd8379e012" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42951 } } }
I20260810 07:15:25.626511 21464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:25.626533 21453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:25.628985 21464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:25.634057 21464 catalog_manager.cc:1347] Generated new cluster ID: 02fe8e64b2cc4dff97b95c2ffcb4a5d6
I20260810 07:15:25.634107 21464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:25.645207 21464 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:25.645983 21464 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:25.649799 21464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f71c52541c42aab579abdd8379e012: Generated new TSK 0
I20260810 07:15:25.650316 21464 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:15:25.677207 21491 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 07:15:25.677330 21491 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:15:25.677911 21446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.660365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53330 (local address 127.12.117.190:42951)
0810 07:15:25.660482 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.660487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.660657 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:15:25.660824 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.660827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.660878 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.660982 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:25.661170 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.661988 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.662787 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.663118 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.663213 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.663518 (+ 305us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:25.677539 (+ 14021us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:15:25.677784 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 07:15:25.677976 21446 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53330: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 07:15:25.679005 21490 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:25.660266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42951 (local address 127.0.0.1:53330)
0810 07:15:25.660587 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.660599 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.660697 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.660982 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.660984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.660990 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:25.661086 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.661090 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.662077 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.662079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.662697 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.662703 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.663209 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.663212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.663305 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.663348 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:25.677721 (+ 14373us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 07:15:25.677887 (+ 166us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.117.190:42951: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.117.190:42951: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45737/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 07:15:25.679632 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 21320
2026-08-10T07:15:25Z chronyd exiting
I20260810 07:15:25.743220 12758 test_util.cc:175] -----------------------------------------------
I20260810 07:15:25.743327 12758 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786346097807571-12758-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (315 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 07:15:25.744388 12758 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:25.745198 12758 webserver.cc:460] Webserver started at https://127.0.0.1:43967/ using document root <none> and password file <none>
I20260810 07:15:25.745294 12758 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:25.745544 12758 webserver.cc:460] Webserver started at http://127.0.0.1:46215/ using document root <none> and password file <none>
2026-08-10T07:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:25Z Disabled control of system clock
I20260810 07:15:25.754142 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36041
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:39683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:36041
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43967/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/testchainca.cert with env {}
W20260810 07:15:25.859081 21498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:25.859315 21498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:25.859368 21498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:25.863181 21498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:25.863227 21498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:25.863248 21498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:25.863286 21498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:25.863309 21498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:25.863337 21498 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:25.863358 21498 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43967/jwks.json
I20260810 07:15:25.868337 21498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:39683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:36041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43967/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:25.869202 21498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:25.870080 21498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:25.877810 21510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:25.877839 21508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:25.877976 21498 server_base.cc:1034] running on GCE node
W20260810 07:15:25.877833 21505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:25.878650 21498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:25.879665 21498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:25.880825 21498 hybrid_clock.cc:648] HybridClock initialized: now 1786346125880808 us; error 28 us; skew 500 ppm
I20260810 07:15:25.882494 21498 webserver.cc:460] Webserver started at http://127.12.117.190:45847/ using document root <none> and password file <none>
I20260810 07:15:25.883013 21498 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:25.883077 21498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:25.883251 21498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:25.885142 21498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "264a91f544f4496ebc844012bfceb31f"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "b741a556839e28d81df33ea131947ae8"
server_key_iv: "b3c323c83eabd615aa753b42f1a46e35"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.885676 21498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "264a91f544f4496ebc844012bfceb31f"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "b741a556839e28d81df33ea131947ae8"
server_key_iv: "b3c323c83eabd615aa753b42f1a46e35"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.889086 21498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 07:15:25.891305 21518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:25.892385 21498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:15:25.892496 21498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "264a91f544f4496ebc844012bfceb31f"
format_stamp: "Formatted at 2026-08-10 07:15:25 on dist-test-slave-gjrz"
server_key: "b741a556839e28d81df33ea131947ae8"
server_key_iv: "b3c323c83eabd615aa753b42f1a46e35"
server_key_version: "encryptionkey@0"
I20260810 07:15:25.892596 21498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:25.907639 21498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:25.908232 21498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:25.908390 21498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:25.914644 21498 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:36041
I20260810 07:15:25.914654 21613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:36041 every 8 connection(s)
I20260810 07:15:25.915532 21498 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:25.918668 21615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:25.918763 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 21498
I20260810 07:15:25.918879 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:25.919173 12758 external_mini_cluster.cc:1428] Setting key 9d6b8f7ca9b402f237d9148b1bbe50c2
I20260810 07:15:25.924793 21615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f: Bootstrap starting.
I20260810 07:15:25.926787 21615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:25.927556 21615 log.cc:824] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:25.928045 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.920498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51780 (local address 127.12.117.190:36041)
0810 07:15:25.921107 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.921122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.921155 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:15:25.921904 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.921923 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.922292 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.922586 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:25.922984 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.923962 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.924840 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.925325 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.925480 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.926229 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:25.926234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:25.926246 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:25.926598 (+ 352us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:25.926652 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:25.926671 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:25.926755 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:25.926895 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 07:15:25.927137 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":130,"threads_started":1}
I20260810 07:15:25.929488 21615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f: No bootstrap required, opened a new log
I20260810 07:15:25.930670 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:25.933039 21615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } }
I20260810 07:15:25.933233 21615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:25.933287 21615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264a91f544f4496ebc844012bfceb31f, State: Initialized, Role: FOLLOWER
I20260810 07:15:25.933699 21615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } }
I20260810 07:15:25.933832 21615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:25.933898 21615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:25.934005 21615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:25.934738 21615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } }
I20260810 07:15:25.935019 21615 leader_election.cc:302] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 264a91f544f4496ebc844012bfceb31f; no voters:
I20260810 07:15:25.935240 21615 leader_election.cc:288] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:25.935352 21622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:25.935554 21622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [term 1 LEADER]: Becoming Leader. State: Replica: 264a91f544f4496ebc844012bfceb31f, State: Running, Role: LEADER
I20260810 07:15:25.935842 21622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } }
I20260810 07:15:25.936311 21615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:25.937251 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "264a91f544f4496ebc844012bfceb31f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } } }
I20260810 07:15:25.937390 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:25.937665 21626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 264a91f544f4496ebc844012bfceb31f. Latest consensus state: current_term: 1 leader_uuid: "264a91f544f4496ebc844012bfceb31f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264a91f544f4496ebc844012bfceb31f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36041 } } }
I20260810 07:15:25.937767 21626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:25.937855 21634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:25.940481 21634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:25.945574 21634 catalog_manager.cc:1347] Generated new cluster ID: 1fde338f245145fb978ef7978cbacd85
I20260810 07:15:25.945637 21634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:25.954537 21634 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:25.955367 21634 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:25.962375 21634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 264a91f544f4496ebc844012bfceb31f: Generated new TSK 0
I20260810 07:15:25.962893 21634 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 07:15:25.998824 21668 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 07:15:25.998900 21668 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 07:15:25.999439 21619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:25.976737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.12.117.190:36041)
0810 07:15:25.976859 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:25.976864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:25.977000 (+ 136us) server_negotiation.cc:407] Connection header received
0810 07:15:25.977186 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:25.977189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:25.977235 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:25.977329 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:25.977533 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.978093 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.979311 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.979531 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.979605 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.979870 (+ 265us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:25.999109 (+ 19239us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43967/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:15:25.999336 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51790: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43967/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:15:25.999493 21619 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51790: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43967/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 07:15:25.999591 21665 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:25.976624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:36041 (local address 127.0.0.1:51790)
0810 07:15:25.976912 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:25.976926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:25.977044 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:25.977330 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:25.977333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:25.977341 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:25.977446 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.977452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.978192 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.978196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.979185 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:25.979195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:25.979624 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:25.979627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:25.979718 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:25.979743 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:25.999253 (+ 19510us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43967/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 07:15:25.999427 (+ 174us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.117.190:36041: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43967/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":108,"threads_started":1}
I20260810 07:15:26.000207 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 21498
2026-08-10T07:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 07:15:26.044543 12758 mini_oidc.cc:152] Starting JWKS server
I20260810 07:15:26.045398 12758 webserver.cc:460] Webserver started at https://127.0.0.1:45937/ using document root <none> and password file <none>
I20260810 07:15:26.045496 12758 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 07:15:26.045743 12758 webserver.cc:460] Webserver started at http://127.0.0.1:39319/ using document root <none> and password file <none>
2026-08-10T07:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:26Z Disabled control of system clock
I20260810 07:15:26.054916 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:32779
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:32779
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45937/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:15:26.160661 21677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:26.160895 21677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:26.160943 21677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:26.164811 21677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:26.164862 21677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:26.164884 21677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:26.164904 21677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:26.164924 21677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:26.164953 21677 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:26.164973 21677 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45937/jwks.json
I20260810 07:15:26.169540 21677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:32779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:32779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45937/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:26.170456 21677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:26.171739 21677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:26.179553 21692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:26.179677 21677 server_base.cc:1034] running on GCE node
W20260810 07:15:26.179585 21691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:26.179585 21696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:26.180397 21677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:26.181373 21677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:26.182523 21677 hybrid_clock.cc:648] HybridClock initialized: now 1786346126182498 us; error 37 us; skew 500 ppm
I20260810 07:15:26.189096 21677 webserver.cc:460] Webserver started at http://127.12.117.190:37959/ using document root <none> and password file <none>
I20260810 07:15:26.189638 21677 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:26.189698 21677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:26.189868 21677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:26.191874 21677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "02a653f4e9fd4350a83bcf5c6232bd07"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f513fe92d9c477a8ddebee1584cface3"
server_key_iv: "87d703e8269ee021a1593d20576d9a50"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.192389 21677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "02a653f4e9fd4350a83bcf5c6232bd07"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f513fe92d9c477a8ddebee1584cface3"
server_key_iv: "87d703e8269ee021a1593d20576d9a50"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.195757 21677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:15:26.197976 21707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:26.199095 21677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:26.199203 21677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "02a653f4e9fd4350a83bcf5c6232bd07"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f513fe92d9c477a8ddebee1584cface3"
server_key_iv: "87d703e8269ee021a1593d20576d9a50"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.199335 21677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:26.217231 21677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:26.217784 21677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:26.217918 21677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:26.224308 21677 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:32779
I20260810 07:15:26.224320 21798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:32779 every 8 connection(s)
I20260810 07:15:26.225157 21677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:26.227836 21799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:26.229825 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 21677
I20260810 07:15:26.229925 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:26.230158 12758 external_mini_cluster.cc:1428] Setting key df39d4b8f3ee5d82f7c1c43faee586c9
I20260810 07:15:26.233381 21799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07: Bootstrap starting.
I20260810 07:15:26.235743 21799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:26.236497 21799 log.cc:824] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:26.238418 21799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07: No bootstrap required, opened a new log
I20260810 07:15:26.238770 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.231419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33410 (local address 127.12.117.190:32779)
0810 07:15:26.231966 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.231980 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.232014 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:15:26.232620 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.232635 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.232997 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.233263 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:26.233616 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.234550 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.235750 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.236226 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.236339 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.237153 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:26.237159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:26.237175 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:26.237538 (+ 363us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:26.237591 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:26.237611 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:26.237690 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:26.237846 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:15:26.238100 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":100,"threads_started":1}
I20260810 07:15:26.241410 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:26.242259 21799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } }
I20260810 07:15:26.242468 21799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:26.242518 21799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a653f4e9fd4350a83bcf5c6232bd07, State: Initialized, Role: FOLLOWER
I20260810 07:15:26.242941 21799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } }
I20260810 07:15:26.243103 21799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:26.243167 21799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:26.243283 21799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:26.244045 21799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } }
I20260810 07:15:26.244335 21799 leader_election.cc:302] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02a653f4e9fd4350a83bcf5c6232bd07; no voters:
I20260810 07:15:26.244593 21799 leader_election.cc:288] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:26.244675 21808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:26.244874 21808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [term 1 LEADER]: Becoming Leader. State: Replica: 02a653f4e9fd4350a83bcf5c6232bd07, State: Running, Role: LEADER
I20260810 07:15:26.245215 21808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } }
I20260810 07:15:26.245668 21799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:26.246711 21809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } } }
I20260810 07:15:26.246835 21809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:26.247211 21826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:26.247190 21810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02a653f4e9fd4350a83bcf5c6232bd07. Latest consensus state: current_term: 1 leader_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a653f4e9fd4350a83bcf5c6232bd07" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 32779 } } }
I20260810 07:15:26.247826 21810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:26.250025 21826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:26.255229 21826 catalog_manager.cc:1347] Generated new cluster ID: f961b093571a414f9051e61cf45a089f
I20260810 07:15:26.255302 21826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:26.269932 21826 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:26.271072 21826 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:26.276099 21826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02a653f4e9fd4350a83bcf5c6232bd07: Generated new TSK 0
I20260810 07:15:26.276770 21826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:26.293392 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.288751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.12.117.190:32779)
0810 07:15:26.288864 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.288868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.289005 (+ 137us) server_negotiation.cc:407] Connection header received
0810 07:15:26.289143 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.289146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.289199 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.289304 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:26.289447 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.290252 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.291048 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.291423 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.291514 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.291765 (+ 251us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:26.293064 (+ 1299us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:15:26.293274 (+ 210us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:15:26.293530 21805 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33420: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 07:15:26.295006 21853 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:26.288666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:32779 (local address 127.0.0.1:33420)
0810 07:15:26.288931 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:26.288943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:26.289037 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:26.289293 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:26.289295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:26.289300 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:26.289382 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.289387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.290336 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.290339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.290957 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.290963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.291542 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.291545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.291607 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.291628 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:26.293220 (+ 1592us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 07:15:26.294845 (+ 1625us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.117.190:32779: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45937/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":1463,"thread_start_us":96,"threads_started":1}
I20260810 07:15:26.295615 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 21677
E20260810 07:15:26.311707 21854 webserver.cc:533] Webserver: sslize failed:
E20260810 07:15:26.311764 21854 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T07:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (297 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T07:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:26Z Disabled control of system clock
I20260810 07:15:26.349669 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38317
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:38317
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/testchainca.cert with env {}
W20260810 07:15:26.453323 21862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:26.453576 21862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:26.453619 21862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:26.457442 21862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:26.457487 21862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:26.457501 21862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:26.457520 21862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:26.457535 21862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 07:15:26.457548 21862 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 07:15:26.457559 21862 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 07:15:26.462455 21862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:38317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:26.463297 21862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:26.464195 21862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:26.471807 21877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:26.471802 21872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:26.471802 21879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:26.471853 21862 server_base.cc:1034] running on GCE node
I20260810 07:15:26.472553 21862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:26.473489 21862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:26.474650 21862 hybrid_clock.cc:648] HybridClock initialized: now 1786346126474623 us; error 38 us; skew 500 ppm
I20260810 07:15:26.476264 21862 webserver.cc:460] Webserver started at http://127.12.117.190:34987/ using document root <none> and password file <none>
I20260810 07:15:26.476768 21862 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:26.476830 21862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:26.476998 21862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:26.478807 21862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "d9c31c0d47e246bebb73709f2ce68084"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f8a3070b458b265425c00a1316e9070d"
server_key_iv: "59ee4dec28e573d2d5c83056e860e1d8"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.479357 21862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "d9c31c0d47e246bebb73709f2ce68084"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f8a3070b458b265425c00a1316e9070d"
server_key_iv: "59ee4dec28e573d2d5c83056e860e1d8"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.482628 21862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:26.484771 21889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:26.485841 21862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:26.485946 21862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "d9c31c0d47e246bebb73709f2ce68084"
format_stamp: "Formatted at 2026-08-10 07:15:26 on dist-test-slave-gjrz"
server_key: "f8a3070b458b265425c00a1316e9070d"
server_key_iv: "59ee4dec28e573d2d5c83056e860e1d8"
server_key_version: "encryptionkey@0"
I20260810 07:15:26.486033 21862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:26.497191 21862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:26.497746 21862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:26.497896 21862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:26.504141 21862 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:38317
I20260810 07:15:26.504159 22012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:38317 every 8 connection(s)
I20260810 07:15:26.504971 21862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:26.507906 22014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:26.512893 22014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084: Bootstrap starting.
I20260810 07:15:26.514318 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 21862
I20260810 07:15:26.514429 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:26.514675 12758 external_mini_cluster.cc:1428] Setting key d2892d216fa10c7e0fea20393cc32d27
I20260810 07:15:26.515188 22014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:26.515915 22014 log.cc:824] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:26.517624 22014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084: No bootstrap required, opened a new log
I20260810 07:15:26.520840 22014 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } }
I20260810 07:15:26.521036 22014 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:26.521091 22014 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c31c0d47e246bebb73709f2ce68084, State: Initialized, Role: FOLLOWER
I20260810 07:15:26.521497 22014 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } }
I20260810 07:15:26.521623 22014 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:26.521687 22014 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:26.521783 22014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:26.522688 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.515605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.12.117.190:38317)
0810 07:15:26.516202 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.516222 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.516264 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:15:26.516856 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.516871 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.517170 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.517451 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:26.517847 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.518781 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.519680 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.520122 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.520261 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.521047 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:26.521051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:26.521067 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:26.521443 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:26.521497 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:26.521526 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:26.521610 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:26.521748 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:15:26.521984 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":93,"threads_started":1}
I20260810 07:15:26.522686 22014 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } }
I20260810 07:15:26.523030 22014 leader_election.cc:302] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9c31c0d47e246bebb73709f2ce68084; no voters:
I20260810 07:15:26.523310 22014 leader_election.cc:288] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:26.523392 22020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:26.523564 22020 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [term 1 LEADER]: Becoming Leader. State: Replica: d9c31c0d47e246bebb73709f2ce68084, State: Running, Role: LEADER
I20260810 07:15:26.523896 22020 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } }
I20260810 07:15:26.524483 22014 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:26.525363 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:15:26.525420 22022 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9c31c0d47e246bebb73709f2ce68084. Latest consensus state: current_term: 1 leader_uuid: "d9c31c0d47e246bebb73709f2ce68084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } } }
I20260810 07:15:26.525532 22022 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:26.525835 22021 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9c31c0d47e246bebb73709f2ce68084" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9c31c0d47e246bebb73709f2ce68084" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38317 } } }
I20260810 07:15:26.525913 22021 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:26.527122 22036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:26.529677 22036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:26.534312 22036 catalog_manager.cc:1347] Generated new cluster ID: 29343330340a464eb775d27f2500bc4e
I20260810 07:15:26.534373 22036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:26.547484 22036 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:26.548259 22036 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:26.556298 22036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9c31c0d47e246bebb73709f2ce68084: Generated new TSK 0
I20260810 07:15:26.556849 22036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:26.581660 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:26.566135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48888 (local address 127.12.117.190:38317)
0810 07:15:26.566242 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:26.566246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:26.566417 (+ 171us) server_negotiation.cc:407] Connection header received
0810 07:15:26.566595 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:26.566597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:26.566640 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:26.566723 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0810 07:15:26.566993 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.567559 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.568397 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.568611 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.568695 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.568991 (+ 296us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 07:15:26.581312 (+ 12321us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:15:26.581538 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:15:26.581729 22017 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 07:15:26.581782 22061 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:26.566048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38317 (local address 127.0.0.1:48888)
0810 07:15:26.566333 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:26.566345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:26.566469 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:26.566734 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:26.566737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:26.566745 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 07:15:26.566872 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.566877 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.567669 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.567673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.568290 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:26.568298 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:26.568722 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:26.568726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:26.568817 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:26.568838 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 07:15:26.581486 (+ 12648us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 07:15:26.581625 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.117.190:38317: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
I20260810 07:15:26.582636 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 21862
2026-08-10T07:15:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (267 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (117 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (116 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:26 dist-test-slave-gjrz krb5kdc[22085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:26 dist-test-slave-gjrz krb5kdc[22085](info): set up 2 sockets
Aug 10 07:15:26 dist-test-slave-gjrz krb5kdc[22085](info): commencing operation
krb5kdc: starting...
W20260810 07:15:27.977681 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:28.096359 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37055
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:28.204619 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:28.204859 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:28.204905 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:28.208837 22101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:28.208889 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:28.208907 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:28.208921 22101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:28.208935 22101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:28.214085 22101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:28.215010 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:28.215946 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:28.224694 22115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.224695 22113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.224694 22111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.224938 22101 server_base.cc:1034] running on GCE node
I20260810 07:15:28.225449 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:28.226568 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:28.227722 22101 hybrid_clock.cc:648] HybridClock initialized: now 1786346128227701 us; error 34 us; skew 500 ppm
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:28.230294 22101 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:28.231223 22101 webserver.cc:460] Webserver started at http://127.12.117.190:44041/ using document root <none> and password file <none>
I20260810 07:15:28.231770 22101 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:28.231837 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:28.232026 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:28.233987 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "046ac08708a04f71be42d2372fb1eef4"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "439aaa536f413bd3270c5c60194e0f28"
server_key_iv: "a809319613e43d3d1825b90705eb5c67"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.234534 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "046ac08708a04f71be42d2372fb1eef4"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "439aaa536f413bd3270c5c60194e0f28"
server_key_iv: "a809319613e43d3d1825b90705eb5c67"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.238094 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 07:15:28.240425 22128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.241637 22101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:28.241765 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "046ac08708a04f71be42d2372fb1eef4"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "439aaa536f413bd3270c5c60194e0f28"
server_key_iv: "a809319613e43d3d1825b90705eb5c67"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.241860 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:28.263018 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:28.265779 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:28.265949 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:28.272837 22101 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37055
I20260810 07:15:28.272872 22226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37055 every 8 connection(s)
I20260810 07:15:28.273746 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:28.276492 22228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:28.281756 22228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4: Bootstrap starting.
I20260810 07:15:28.281986 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 22101
I20260810 07:15:28.282099 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:28.282362 12758 external_mini_cluster.cc:1428] Setting key 69b08079456b11f90d26764a33642502
I20260810 07:15:28.284255 22228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:28.284940 22228 log.cc:824] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:28.286684 22228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4: No bootstrap required, opened a new log
I20260810 07:15:28.290148 22228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } }
I20260810 07:15:28.290410 22228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:28.290479 22228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046ac08708a04f71be42d2372fb1eef4, State: Initialized, Role: FOLLOWER
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:28.290966 22228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } }
I20260810 07:15:28.291100 22228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:28.291157 22228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:28.291249 22228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:28.292208 22228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } }
I20260810 07:15:28.292550 22228 leader_election.cc:302] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 046ac08708a04f71be42d2372fb1eef4; no voters:
I20260810 07:15:28.292836 22228 leader_election.cc:288] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:28.292918 22234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:28.293121 22234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [term 1 LEADER]: Becoming Leader. State: Replica: 046ac08708a04f71be42d2372fb1eef4, State: Running, Role: LEADER
I20260810 07:15:28.293479 22234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } }
I20260810 07:15:28.294086 22228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:28.295064 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 046ac08708a04f71be42d2372fb1eef4. Latest consensus state: current_term: 1 leader_uuid: "046ac08708a04f71be42d2372fb1eef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } } }
I20260810 07:15:28.295188 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:28.295557 22235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "046ac08708a04f71be42d2372fb1eef4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "046ac08708a04f71be42d2372fb1eef4" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37055 } } }
I20260810 07:15:28.295691 22235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:28.296520 22246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:28.297063 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.283652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.12.117.190:37055)
0810 07:15:28.284082 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.284097 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.284133 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:15:28.284799 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.284819 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.285161 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.285476 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.286296 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.287330 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.288202 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.288685 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.288845 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.291485 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:28.291490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:28.291503 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:28.291530 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:28.293264 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.293773 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.293777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.293782 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.293929 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.294250 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.294253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.294254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.294751 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:28.295662 (+ 911us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:28.295832 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:15:28.296126 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":131,"threads_started":1}
I20260810 07:15:28.299746 22246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:28.305284 22246 catalog_manager.cc:1347] Generated new cluster ID: 8eb99e90b4894e23816263eeecdd76d0
I20260810 07:15:28.305334 22246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:28.318491 22246 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:28.319809 22246 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:28.344492 22246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 046ac08708a04f71be42d2372fb1eef4: Generated new TSK 0
I20260810 07:15:28.345300 22246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:28.380223 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:28.507392 22261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:28.507630 22261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:28.507688 22261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:28.511631 22261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:28.511680 22261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:28.511754 22261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:28.516731 22261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:28.517679 22261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:28.518654 22261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:28.527626 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.527626 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.527663 22261 server_base.cc:1034] running on GCE node
W20260810 07:15:28.527626 22273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.528236 22261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:28.528905 22261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:28.530108 22261 hybrid_clock.cc:648] HybridClock initialized: now 1786346128530093 us; error 34 us; skew 500 ppm
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:28.532672 22261 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:28.533425 22261 webserver.cc:460] Webserver started at http://127.12.117.129:33657/ using document root <none> and password file <none>
I20260810 07:15:28.533892 22261 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:28.533947 22261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:28.534114 22261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:28.536010 22261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "153792b995d64deca44f232263bd50f9"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "cf264d22acab00d6276e82312450b8a5"
server_key_iv: "e500df43d984e5fe170543d2648aba3f"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.536509 22261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "153792b995d64deca44f232263bd50f9"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "cf264d22acab00d6276e82312450b8a5"
server_key_iv: "e500df43d984e5fe170543d2648aba3f"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.540000 22261 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:28.542235 22293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.543377 22261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:28.543469 22261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "153792b995d64deca44f232263bd50f9"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "cf264d22acab00d6276e82312450b8a5"
server_key_iv: "e500df43d984e5fe170543d2648aba3f"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.543548 22261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:28.551714 22261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:28.554112 22261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:28.554248 22261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:28.554706 22261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:28.555493 22261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:28.555541 22261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.555578 22261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:28.555598 22261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.563850 22261 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38749
I20260810 07:15:28.563882 22499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38749 every 8 connection(s)
I20260810 07:15:28.564630 22261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:28.568207 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 22261
I20260810 07:15:28.568336 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:28.568650 12758 external_mini_cluster.cc:1428] Setting key e50c670886812afc0d44a81b0e7a928f
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:28.578202 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.566215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:58269 (local address 127.12.117.190:37055)
0810 07:15:28.566337 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.566340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.567118 (+ 778us) server_negotiation.cc:407] Connection header received
0810 07:15:28.568531 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.568534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.568583 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.568659 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.569904 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.570465 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.572043 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.572399 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.572501 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.574697 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:28.574702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:28.574705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:28.574735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:28.576249 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.576715 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.576719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.576721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.576822 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.577120 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.577123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.577125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.577358 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:28.577466 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:28.577886 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 07:15:28.578016 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:28.578809 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.566534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37055 (local address 127.12.117.129:58269)
0810 07:15:28.566947 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.567010 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.568310 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.568830 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.568839 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.569263 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.569739 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.569753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.570596 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.570600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.571894 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.571909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.572543 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.572546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.572679 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.573177 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.573192 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.574504 (+ 1312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:28.576362 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.576367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.576378 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.576618 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.576902 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.576903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.576905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.577027 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.577451 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:28.577455 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:28.577661 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 07:15:28.577804 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 07:15:28.578036 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":121,"threads_started":1}
I20260810 07:15:28.579854 22501 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37055
I20260810 07:15:28.580066 22501 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:28.580547 22501 heartbeater.cc:511] Master 127.12.117.190:37055 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:15:28.582048 22151 ts_manager.cc:194] Registered new tserver with Master: 153792b995d64deca44f232263bd50f9 (127.12.117.129:38749)
I20260810 07:15:28.583148 22151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:58269
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:28.595185 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:28.704372 22510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:28.704651 22510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:28.704710 22510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:28.708657 22510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:28.708706 22510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:28.708783 22510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:28.713776 22510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:28.714713 22510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:28.715691 22510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:28.724732 22522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.724840 22528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.724727 22524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.724952 22510 server_base.cc:1034] running on GCE node
I20260810 07:15:28.725384 22510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:28.726013 22510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:28.727178 22510 hybrid_clock.cc:648] HybridClock initialized: now 1786346128727159 us; error 37 us; skew 500 ppm
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:28.729836 22510 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:28.730681 22510 webserver.cc:460] Webserver started at http://127.12.117.130:46521/ using document root <none> and password file <none>
I20260810 07:15:28.731184 22510 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:28.731240 22510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:28.731456 22510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:28.733371 22510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "1d557822d8d648a084b68a425c78a827"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "5f3b6d652629db10ae8c83849bae926b"
server_key_iv: "3428f20f9b230e6fcce699ea6077b3af"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.733898 22510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "1d557822d8d648a084b68a425c78a827"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "5f3b6d652629db10ae8c83849bae926b"
server_key_iv: "3428f20f9b230e6fcce699ea6077b3af"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.737470 22510 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:28.739841 22541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.741019 22510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:28.741123 22510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "1d557822d8d648a084b68a425c78a827"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "5f3b6d652629db10ae8c83849bae926b"
server_key_iv: "3428f20f9b230e6fcce699ea6077b3af"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.741218 22510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:28.757027 22510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:28.759918 22510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:28.760080 22510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:28.760600 22510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:28.761385 22510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:28.761431 22510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.761472 22510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:28.761502 22510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.769501 22510 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:43645
I20260810 07:15:28.769562 22747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:43645 every 8 connection(s)
I20260810 07:15:28.770299 22510 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:28.771109 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 22510
I20260810 07:15:28.771209 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:28.771536 12758 external_mini_cluster.cc:1428] Setting key 7511474f0c03f13a84a6a9aeb184b841
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:15:28.783962 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.772048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:55861 (local address 127.12.117.190:37055)
0810 07:15:28.772198 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.772202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.773146 (+ 944us) server_negotiation.cc:407] Connection header received
0810 07:15:28.774784 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.774787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.774832 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.774891 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.776128 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.776820 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.777992 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.778224 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.778315 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.780667 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:28.780671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:28.780673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:28.780696 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:28.781966 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.782432 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.782436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.782438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.782520 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.782839 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.782843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.782844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.783019 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:28.783118 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:28.783682 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 07:15:28.783804 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 07:15:28.784713 22751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.772451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37055 (local address 127.12.117.130:55861)
0810 07:15:28.772980 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.773037 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.774550 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.775082 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.775092 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.775506 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.775974 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.775988 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.776975 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.776978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.777862 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.777872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.778370 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.778374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.778557 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.779097 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.779115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.780506 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:28.782076 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.782082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.782096 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.782331 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.782603 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.782614 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.782615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.782748 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.783121 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:28.783128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:28.783422 (+ 294us) client_negotiation.cc:769] Sending connection context
0810 07:15:28.783626 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 07:15:28.783990 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":161,"threads_started":1}
I20260810 07:15:28.785826 22748 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37055
I20260810 07:15:28.786042 22748 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:28.786475 22748 heartbeater.cc:511] Master 127.12.117.190:37055 requested a full tablet report, sending...
I20260810 07:15:28.787528 22151 ts_manager.cc:194] Registered new tserver with Master: 1d557822d8d648a084b68a425c78a827 (127.12.117.130:43645)
I20260810 07:15:28.788070 22151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:55861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:28.799319 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:28.907639 22755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:28.907855 22755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:28.907893 22755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:28.911782 22755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:28.911825 22755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:28.911876 22755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:28.916932 22755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:28.917755 22755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:28.918599 22755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:28.927663 22764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:28.927714 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.927734 22755 server_base.cc:1034] running on GCE node
W20260810 07:15:28.927870 22765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:28.928264 22755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:28.928808 22755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:28.929965 22755 hybrid_clock.cc:648] HybridClock initialized: now 1786346128929947 us; error 27 us; skew 500 ppm
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:28.932523 22755 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:28.933339 22755 webserver.cc:460] Webserver started at http://127.12.117.131:37629/ using document root <none> and password file <none>
I20260810 07:15:28.933826 22755 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:28.933892 22755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:28.934082 22755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:28.935999 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "7b092e0986db4b3eae38f70f701b86c7"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "a32f9ce2018c3996f5c9412f5a57ea97"
server_key_iv: "858e7487da8a44655c5970db02029ac7"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.936628 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "7b092e0986db4b3eae38f70f701b86c7"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "a32f9ce2018c3996f5c9412f5a57ea97"
server_key_iv: "858e7487da8a44655c5970db02029ac7"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.940238 22755 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:28.942454 22789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.943593 22755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:28.943707 22755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "7b092e0986db4b3eae38f70f701b86c7"
format_stamp: "Formatted at 2026-08-10 07:15:28 on dist-test-slave-gjrz"
server_key: "a32f9ce2018c3996f5c9412f5a57ea97"
server_key_iv: "858e7487da8a44655c5970db02029ac7"
server_key_version: "encryptionkey@0"
I20260810 07:15:28.943802 22755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:28.958884 22755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:28.961699 22755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:28.961875 22755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:28.962396 22755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:28.963160 22755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:28.963208 22755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.963280 22755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:28.963312 22755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:28.971323 22755 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:45081
I20260810 07:15:28.971377 22978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:45081 every 8 connection(s)
I20260810 07:15:28.972127 22755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:28.975116 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 22755
I20260810 07:15:28.975225 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:28.975510 12758 external_mini_cluster.cc:1428] Setting key 8905b6c82ba613bcdfe36b05707dc0bd
Aug 10 07:15:28 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346128, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:28.984493 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.973695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:37877 (local address 127.12.117.190:37055)
0810 07:15:28.973814 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:28.973818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:28.974572 (+ 754us) server_negotiation.cc:407] Connection header received
0810 07:15:28.976018 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:28.976021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:28.976076 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:28.976165 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:28.977525 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.978100 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.979183 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.979461 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.979539 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.981622 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:28.981625 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:28.981627 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:28.981650 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:28.982872 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.983313 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.983316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.983317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.983399 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:28.983682 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:28.983684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:28.983685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:28.983842 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:28.983910 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:28.984292 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 07:15:28.984379 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:28.985158 22985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:28.974016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37055 (local address 127.12.117.131:37877)
0810 07:15:28.974426 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:28.974466 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:28.975748 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:28.976351 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:28.976360 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:28.976778 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:28.977377 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.977398 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.978210 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.978212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.979075 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:28.979084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:28.979568 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:28.979571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:28.979716 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:28.980147 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:28.980164 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:28.981472 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:28.982958 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.982962 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.982971 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.983195 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.983476 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:28.983478 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:28.983479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:28.983604 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:28.983912 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:28.983917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:28.984113 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 07:15:28.984256 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 07:15:28.984465 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":108,"threads_started":1}
I20260810 07:15:28.986133 22982 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37055
I20260810 07:15:28.986333 22982 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:28.986752 22982 heartbeater.cc:511] Master 127.12.117.190:37055 requested a full tablet report, sending...
I20260810 07:15:28.987756 22151 ts_manager.cc:194] Registered new tserver with Master: 7b092e0986db4b3eae38f70f701b86c7 (127.12.117.131:45081)
I20260810 07:15:28.988497 22151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:37877
I20260810 07:15:28.989391 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/test-admin.keytab.
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.004539 12758 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 07:15:29.007055 22993 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.005667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57632 (local address 127.12.117.129:38749)
0810 07:15:29.006023 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.006032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.006054 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:15:29.006098 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.006102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.006254 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.006361 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.006857 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57632: BlockingRecv error: recv got EOF from 127.0.0.1:57632 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":154,"threads_started":1}
W20260810 07:15:29.007099 12758 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.008045 12758 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.008888 12758 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:29.017782 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.009662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57636 (local address 127.12.117.129:38749)
0810 07:15:29.009774 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.009778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.009837 (+ 59us) server_negotiation.cc:407] Connection header received
0810 07:15:29.009971 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.009973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.010010 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.010069 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.010579 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.011383 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.012201 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.012547 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.012652 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.014687 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:29.014696 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:29.014706 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:29.014729 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:29.016088 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.016490 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.016494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.016496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.016573 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.016818 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.016821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.016823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.017058 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:29.017244 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:29.017471 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 07:15:29.017621 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:15:29.021014 22993 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.020161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.12.117.129:38749)
0810 07:15:29.020275 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.020279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.020346 (+ 67us) server_negotiation.cc:407] Connection header received
0810 07:15:29.020453 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.020456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.020493 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.020555 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.020928 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57646: BlockingRecv error: recv got EOF from 127.0.0.1:57646 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 07:15:29.021122 12758 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.022006 12758 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.022765 12758 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:29.029920 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.023503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.12.117.129:38749)
0810 07:15:29.023622 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.023624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.023683 (+ 59us) server_negotiation.cc:407] Connection header received
0810 07:15:29.023765 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.023767 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.023798 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.023857 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.024247 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.024726 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.025394 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.025598 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.025663 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.027577 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:29.027580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:29.027582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:29.027602 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:29.028628 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.028961 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.028964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.028965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.029026 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.029202 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.029203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.029204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.029336 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:29.029421 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:29.029677 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 07:15:29.029789 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:15:29.032213 22993 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:29.031340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.12.117.129:38749)
0810 07:15:29.031449 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.031453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.031547 (+ 94us) server_negotiation.cc:407] Connection header received
0810 07:15:29.031646 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.031649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.031686 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.031749 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.032129 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57668: BlockingRecv error: recv got EOF from 127.0.0.1:57668 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:15:29.032279 12758 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786346097807571-12758-0/krb5kdc/krb5cc' of type FILE
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.033094 12758 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:29.033856 12758 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[22085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:15:29.040962 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:29.034546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57680 (local address 127.12.117.129:38749)
0810 07:15:29.034648 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:29.034652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:29.034750 (+ 98us) server_negotiation.cc:407] Connection header received
0810 07:15:29.034850 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:29.034853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:29.034887 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:29.034947 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:29.035451 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.035897 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.036619 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:29.036820 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:29.036902 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:29.038738 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:29.038741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:29.038743 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:29.038763 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:29.039848 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.040207 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.040209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.040210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.040273 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:29.040447 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:29.040448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:29.040449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:29.040573 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:29.040670 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:29.040798 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 07:15:29.040872 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:29.041697 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 22261
I20260810 07:15:29.056663 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 22510
I20260810 07:15:29.065833 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 22755
I20260810 07:15:29.074819 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 22101
2026-08-10T07:15:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2255 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[23012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[23012](info): set up 2 sockets
Aug 10 07:15:29 dist-test-slave-gjrz krb5kdc[23012](info): commencing operation
krb5kdc: starting...
W20260810 07:15:30.235137 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab.
Entry for principal oryx/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab.
I20260810 07:15:30.347944 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:41229
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab
--principal=oryx/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:30.452051 23034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:30.452267 23034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:30.452302 23034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:30.456104 23034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:30.456146 23034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:30.456162 23034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:30.456182 23034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:30.456195 23034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:30.461082 23034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab
--principal=oryx/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:30.461916 23034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:30.462766 23034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:30.470587 23055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:30.470582 23048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:30.470707 23034 server_base.cc:1034] running on GCE node
W20260810 07:15:30.470594 23051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:30.471434 23034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:30.472441 23034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:30.473598 23034 hybrid_clock.cc:648] HybridClock initialized: now 1786346130473584 us; error 24 us; skew 500 ppm
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:30.476174 23034 init.cc:377] Logged in from keytab as oryx/127.12.117.190@KRBTEST.COM (short username oryx)
I20260810 07:15:30.476946 23034 webserver.cc:460] Webserver started at http://127.12.117.190:39291/ using document root <none> and password file <none>
I20260810 07:15:30.477418 23034 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:30.477473 23034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:30.477630 23034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:30.479501 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "f8ae3d13348644b983abe5432ea684dc"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "8693e97e94c61fb6ddb77fd6f7532532"
server_key_iv: "67ce3829951a18935c3d4bcafea8ef40"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.479988 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "f8ae3d13348644b983abe5432ea684dc"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "8693e97e94c61fb6ddb77fd6f7532532"
server_key_iv: "67ce3829951a18935c3d4bcafea8ef40"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.483392 23034 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:30.485560 23068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.486616 23034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:30.486717 23034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "f8ae3d13348644b983abe5432ea684dc"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "8693e97e94c61fb6ddb77fd6f7532532"
server_key_iv: "67ce3829951a18935c3d4bcafea8ef40"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.486811 23034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:30.498788 23034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:30.501525 23034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:30.501682 23034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:30.507889 23034 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:41229
I20260810 07:15:30.508111 23158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:41229 every 8 connection(s)
I20260810 07:15:30.508754 23034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:30.511212 23160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:30.512621 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 23034
I20260810 07:15:30.512729 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:30.512979 12758 external_mini_cluster.cc:1428] Setting key acb9c354beec359cf79d55fcdd790f18
I20260810 07:15:30.517221 23160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Bootstrap starting.
I20260810 07:15:30.519606 23160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:30.520352 23160 log.cc:824] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:30.522053 23160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: No bootstrap required, opened a new log
I20260810 07:15:30.526209 23160 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } }
I20260810 07:15:30.526448 23160 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:30.526506 23160 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8ae3d13348644b983abe5432ea684dc, State: Initialized, Role: FOLLOWER
I20260810 07:15:30.526537 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.514204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45570 (local address 127.12.117.190:41229)
0810 07:15:30.514590 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.514608 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.514650 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:15:30.515454 (+ 804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.515477 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.515868 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.516224 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.516828 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.518237 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.519054 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.519734 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.519893 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.522009 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:30.522015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:30.522037 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:30.522067 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:30.523827 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.524213 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.524218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.524224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.524352 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.524567 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.524571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.524573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.525028 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:30.525212 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:30.525388 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 07:15:30.525662 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":102,"threads_started":1}
I20260810 07:15:30.526962 23160 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } }
I20260810 07:15:30.527097 23160 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:30.527163 23160 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:30.527257 23160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:30.528219 23160 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } }
I20260810 07:15:30.528571 23160 leader_election.cc:302] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8ae3d13348644b983abe5432ea684dc; no voters:
I20260810 07:15:30.528872 23160 leader_election.cc:288] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:30.528977 23169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:30.529165 23169 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 LEADER]: Becoming Leader. State: Replica: f8ae3d13348644b983abe5432ea684dc, State: Running, Role: LEADER
I20260810 07:15:30.529501 23169 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } }
I20260810 07:15:30.529938 23160 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:30.530982 23172 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } }
I20260810 07:15:30.531054 23173 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ae3d13348644b983abe5432ea684dc. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } }
I20260810 07:15:30.531191 23172 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:30.531232 23173 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:30.531664 23182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:30.534658 23182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:30.539927 23182 catalog_manager.cc:1347] Generated new cluster ID: 80aca1c9dc494a7ebde5fc8c678672eb
I20260810 07:15:30.539988 23182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:30.558701 23182 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:30.559508 23182 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:30.567811 23182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Generated new TSK 0
I20260810 07:15:30.568351 23182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
I20260810 07:15:30.594130 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:30.700575 23196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:30.700801 23196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:30.700843 23196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:30.704739 23196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:30.704787 23196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:30.704856 23196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:30.709882 23196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:30.710762 23196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:30.711674 23196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:30.720347 23203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:30.720435 23208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:30.720371 23214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:30.720502 23196 server_base.cc:1034] running on GCE node
I20260810 07:15:30.720943 23196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:30.721473 23196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:30.722636 23196 hybrid_clock.cc:648] HybridClock initialized: now 1786346130722621 us; error 27 us; skew 500 ppm
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:30.725082 23196 init.cc:377] Logged in from keytab as oryx/127.12.117.129@KRBTEST.COM (short username oryx)
I20260810 07:15:30.725833 23196 webserver.cc:460] Webserver started at http://127.12.117.129:42961/ using document root <none> and password file <none>
I20260810 07:15:30.726316 23196 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:30.726382 23196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:30.726565 23196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:30.728399 23196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "801f9248a6ea48829543c852775bbd1b"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "2bfce837b824e4cff95a4259d20fff72"
server_key_iv: "3f7aadc1f0af6aa993945fc8a7b21ed4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.728912 23196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "801f9248a6ea48829543c852775bbd1b"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "2bfce837b824e4cff95a4259d20fff72"
server_key_iv: "3f7aadc1f0af6aa993945fc8a7b21ed4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.732476 23196 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:30.734588 23226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.735754 23196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:30.735849 23196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "801f9248a6ea48829543c852775bbd1b"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "2bfce837b824e4cff95a4259d20fff72"
server_key_iv: "3f7aadc1f0af6aa993945fc8a7b21ed4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.735947 23196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:30.744987 23196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:30.747359 23196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:30.747509 23196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:30.747996 23196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:30.748720 23196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:30.748765 23196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.748803 23196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:30.748831 23196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.757150 23196 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38829
I20260810 07:15:30.757175 23408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38829 every 8 connection(s)
I20260810 07:15:30.757899 23196 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:30.759320 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 23196
I20260810 07:15:30.759446 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:30.759732 12758 external_mini_cluster.cc:1428] Setting key 01d6c21d920ecee5d3706873f825d558
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:30.771008 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.759511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:60221 (local address 127.12.117.190:41229)
0810 07:15:30.759645 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.759650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.760507 (+ 857us) server_negotiation.cc:407] Connection header received
0810 07:15:30.761978 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.761982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.762042 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.762122 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.763185 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.763759 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.764865 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.765104 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.765202 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.767660 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:30.767664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:30.767667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:30.767693 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:30.768960 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.769556 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.769560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.769562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.769660 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.770070 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.770074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.770076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.770289 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:30.770370 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:30.770797 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 07:15:30.770890 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.130@KRBTEST.COM" created.
I20260810 07:15:30.772017 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.759898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:60221)
0810 07:15:30.760365 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:30.760415 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:30.761761 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:30.762258 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:30.762265 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:30.762593 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:30.763036 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.763049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.763881 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.763884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.764745 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.764755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.765248 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.765252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.765413 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.765931 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:30.765945 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:30.767505 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:30.769070 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:30.769076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:30.769089 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:30.769435 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:30.769776 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:30.769779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:30.769781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:30.769964 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:30.770381 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:30.770387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:30.770601 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 07:15:30.770763 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 07:15:30.770991 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":124,"threads_started":1}
I20260810 07:15:30.773205 23410 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:30.773459 23410 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:30.773898 23410 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:30.775347 23097 ts_manager.cc:194] Registered new tserver with Master: 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:30.776469 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:60221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
I20260810 07:15:30.785590 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:30.891048 23421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:30.891311 23421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:30.891360 23421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:30.895337 23421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:30.895383 23421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:30.895453 23421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:30.900352 23421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:30.901202 23421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:30.902073 23421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:30.910885 23433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:30.910840 23421 server_base.cc:1034] running on GCE node
W20260810 07:15:30.910806 23435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:30.910843 23431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:30.911455 23421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:30.912003 23421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:30.913158 23421 hybrid_clock.cc:648] HybridClock initialized: now 1786346130913142 us; error 27 us; skew 500 ppm
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:30.915763 23421 init.cc:377] Logged in from keytab as oryx/127.12.117.130@KRBTEST.COM (short username oryx)
I20260810 07:15:30.916498 23421 webserver.cc:460] Webserver started at http://127.12.117.130:43217/ using document root <none> and password file <none>
I20260810 07:15:30.916985 23421 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:30.917042 23421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:30.917215 23421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:30.919044 23421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "490e07200bcc4287bfc4af6e1a9753e3"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "d82359ddef4a240a3e0909318000bcbd"
server_key_iv: "030040d80cc93359c71688c74c3dabd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.919589 23421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "490e07200bcc4287bfc4af6e1a9753e3"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "d82359ddef4a240a3e0909318000bcbd"
server_key_iv: "030040d80cc93359c71688c74c3dabd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.923218 23421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:15:30.925397 23443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.926577 23421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:30.926697 23421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "490e07200bcc4287bfc4af6e1a9753e3"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "d82359ddef4a240a3e0909318000bcbd"
server_key_iv: "030040d80cc93359c71688c74c3dabd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:30.926787 23421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:30.940174 23421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:30.942942 23421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:30.943104 23421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:30.943655 23421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:30.944371 23421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:30.944424 23421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.944474 23421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:30.944504 23421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:30.952292 23421 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:45751
I20260810 07:15:30.952348 23633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:45751 every 8 connection(s)
I20260810 07:15:30.953052 23421 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:30.960685 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 23421
I20260810 07:15:30.960805 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:30.961144 12758 external_mini_cluster.cc:1428] Setting key f20973f7c5600e201423231baa2a9697
Aug 10 07:15:30 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:30.965329 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.954588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:41395 (local address 127.12.117.190:41229)
0810 07:15:30.954740 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:30.954745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:30.955497 (+ 752us) server_negotiation.cc:407] Connection header received
0810 07:15:30.956748 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:30.956751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:30.956797 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:30.956866 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:30.957880 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.958386 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.959485 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.959701 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.959780 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.962155 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:30.962158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:30.962161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:30.962185 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:30.963492 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.963945 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.963949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.963951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.964029 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:30.964328 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:30.964331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:30.964333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:30.964512 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:30.964596 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:30.965060 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 07:15:30.965176 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:15:30.966029 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:30.954921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:41395)
0810 07:15:30.955367 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:30.955409 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:30.956553 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:30.957009 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:30.957016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:30.957324 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:30.957764 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.957777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.958479 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.958482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.959388 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:30.959398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:30.959811 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:30.959813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:30.959949 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:30.960383 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:30.960398 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:30.962001 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:30.963596 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:30.963603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:30.963616 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:30.963850 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:30.964109 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:30.964113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:30.964115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:30.964242 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:30.964608 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:30.964615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:30.964834 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:15:30.965000 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 07:15:30.965287 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":101,"threads_started":1}
I20260810 07:15:30.967108 23635 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:30.967342 23635 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:30.967777 23635 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:30.968758 23097 ts_manager.cc:194] Registered new tserver with Master: 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751)
I20260810 07:15:30.969525 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:41395
WARNING: no policy specified for oryx/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
I20260810 07:15:30.986172 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:31.095664 23646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:31.095884 23646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:31.095959 23646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:31.099849 23646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:31.099900 23646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:31.099972 23646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:31.104974 23646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:31.105854 23646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:31.106755 23646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.115759 23659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.115809 23646 server_base.cc:1034] running on GCE node
W20260810 07:15:31.115832 23658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.115754 23664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.116394 23646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.116931 23646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.118171 23646 hybrid_clock.cc:648] HybridClock initialized: now 1786346131118141 us; error 48 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.120844 23646 init.cc:377] Logged in from keytab as oryx/127.12.117.131@KRBTEST.COM (short username oryx)
I20260810 07:15:31.121645 23646 webserver.cc:460] Webserver started at http://127.12.117.131:38857/ using document root <none> and password file <none>
I20260810 07:15:31.122175 23646 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.122243 23646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.122447 23646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:31.124372 23646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "a89d44b9b3d9664b5f29499d59e51c0a"
server_key_iv: "0c3ba8dff8da984a7833a4794cd744a9"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.124899 23646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "a89d44b9b3d9664b5f29499d59e51c0a"
server_key_iv: "0c3ba8dff8da984a7833a4794cd744a9"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.128461 23646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:31.131100 23679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.132309 23646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:31.132421 23646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "a89d44b9b3d9664b5f29499d59e51c0a"
server_key_iv: "0c3ba8dff8da984a7833a4794cd744a9"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.132520 23646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.154078 23646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.156713 23646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:31.156883 23646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.157429 23646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:31.158203 23646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:31.158250 23646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.158303 23646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:31.158332 23646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.166487 23646 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:43867
I20260810 07:15:31.166519 23878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:43867 every 8 connection(s)
I20260810 07:15:31.167320 23646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:31.171646 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 23646
I20260810 07:15:31.171741 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:31.171988 12758 external_mini_cluster.cc:1428] Setting key 82b76e9399f34c61750363b773cf3620
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:31.180205 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.168990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:60941 (local address 127.12.117.190:41229)
0810 07:15:31.169148 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.169153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.169917 (+ 764us) server_negotiation.cc:407] Connection header received
0810 07:15:31.171158 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.171161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.171209 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.171299 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.172462 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.173064 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.174398 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.174632 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.174721 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.177006 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.177010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.177013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.177040 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.178232 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.178718 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.178722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.178723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.178807 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.179122 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.179125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.179127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.179355 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.179446 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.179913 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.180046 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:15:31.180902 23885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.169340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:60941)
0810 07:15:31.169771 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.169811 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.170964 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.171439 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.171446 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.171818 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.172315 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.172329 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.173192 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.173196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.174272 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.174284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.174768 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.174772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.174931 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.175474 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.175491 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.176847 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.178346 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.178358 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.178372 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.178619 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.178893 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.178897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.178899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.179030 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.179455 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.179462 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.179688 (+ 226us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.179850 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.180160 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":121,"threads_started":1}
I20260810 07:15:31.181934 23880 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:31.182183 23880 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:31.182621 23880 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:31.183583 23097 ts_manager.cc:194] Registered new tserver with Master: bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:31.184131 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:60941
I20260810 07:15:31.185587 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.117.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.189.keytab.
Entry for principal oryx/127.12.117.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.189.keytab.
I20260810 07:15:31.210578 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:43029
--webserver_interface=127.12.117.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.189.keytab
--principal=oryx/127.12.117.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:41229,127.12.117.189:43029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:31.319242 23889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:31.319496 23889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:31.319540 23889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:31.323417 23889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:31.323462 23889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:31.323482 23889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:31.323511 23889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:31.323536 23889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:31.328504 23889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:41229,127.12.117.189:43029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.189.keytab
--principal=oryx/127.12.117.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:43029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.117.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
W20260810 07:15:31.328670 23889 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.117.190:41229,127.12.117.189:43029'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 07:15:31.329401 23889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:31.330340 23889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.338449 23897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.338433 23901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.338541 23889 server_base.cc:1034] running on GCE node
W20260810 07:15:31.338423 23898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.339128 23889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.340186 23889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.341349 23889 hybrid_clock.cc:648] HybridClock initialized: now 1786346131341331 us; error 30 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.346803 23889 init.cc:377] Logged in from keytab as oryx/127.12.117.189@KRBTEST.COM (short username oryx)
I20260810 07:15:31.347895 23889 webserver.cc:460] Webserver started at http://127.12.117.189:37743/ using document root <none> and password file <none>
I20260810 07:15:31.348412 23889 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.348470 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.348671 23889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:31.350540 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/instance:
uuid: "3d88e98d411d41038f21fc4875501737"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f305eacd7ab869664ea52156db1c673e"
server_key_iv: "7776a82d151f636eadfa9d26ffe5c065"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.351064 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal/instance:
uuid: "3d88e98d411d41038f21fc4875501737"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f305eacd7ab869664ea52156db1c673e"
server_key_iv: "7776a82d151f636eadfa9d26ffe5c065"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.354465 23889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:15:31.356660 23911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.357728 23889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:31.357839 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "3d88e98d411d41038f21fc4875501737"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f305eacd7ab869664ea52156db1c673e"
server_key_iv: "7776a82d151f636eadfa9d26ffe5c065"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.357929 23889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.376928 23889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.379330 23889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:31.379480 23889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.385679 23889 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.189:43029
I20260810 07:15:31.385953 24012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.189:43029 every 8 connection(s)
I20260810 07:15:31.386518 23889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
I20260810 07:15:31.389029 24015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:31.392431 24015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:15:31.396112 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 23889
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.189@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:31.405666 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.393991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.12.117.190:41229)
0810 07:15:31.394104 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.394108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.394832 (+ 724us) server_negotiation.cc:407] Connection header received
0810 07:15:31.395709 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.395713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.395776 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.395848 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.397048 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.397735 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.399239 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.399508 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.399609 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.402226 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.402229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.402231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.402257 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.403713 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.404168 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.404171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.404172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.404250 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.404549 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.404552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.404554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.404810 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.404921 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.405442 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.405544 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:15:31.406551 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.396841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41114 (local address 127.12.117.189:43029)
0810 07:15:31.397123 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.397132 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.397153 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:15:31.397281 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.397285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.397432 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.397552 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.398315 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.399431 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.400282 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.400750 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.400862 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.403176 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.403183 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.403195 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.403218 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.404650 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.405081 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.405087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.405088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.405203 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.405452 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.405455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.405456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.405716 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.405901 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.406252 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.406363 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"spinlock_wait_cycles":2432,"thread_start_us":87,"threads_started":1}
I20260810 07:15:31.406553 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.394286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45578)
0810 07:15:31.394697 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.394735 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.395499 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.396015 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.396021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.396362 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.396866 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.396885 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.397862 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.397865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.399119 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.399130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.399638 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.399641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.399787 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.400285 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.400305 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.402056 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.403816 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.403825 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.403832 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.404065 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.404333 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.404336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.404337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.404460 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.404907 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.404911 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.405191 (+ 280us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.405372 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.405650 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":109,"threads_started":1}
I20260810 07:15:31.408325 24015 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.189@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:31.414849 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.408777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.12.117.189:43029)
0810 07:15:31.408866 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.408869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.408880 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:31.408956 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.408958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.408988 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.409046 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.409349 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.409804 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.410430 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.410647 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.410712 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.412457 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.412460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.412462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.412481 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.413492 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.413849 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.413853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.413854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.413917 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.414141 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.414144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.414145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.414288 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.414367 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.414585 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.414713 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":30}
I20260810 07:15:31.414849 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.408716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41128)
0810 07:15:31.408816 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.408829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.408892 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.409065 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.409067 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.409185 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.409272 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.409277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.409881 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.409883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.410358 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.410365 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.410721 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.410724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.410792 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.411303 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.411317 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.412345 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.413587 (+ 1242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.413590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.413592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.413767 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.413992 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.413995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.413997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.414064 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.414372 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.414375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.414450 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.414530 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.414660 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:15:31.417676 24015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Bootstrap starting.
I20260810 07:15:31.419884 24015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:31.420558 24015 log.cc:824] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:31.422236 24015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: No bootstrap required, opened a new log
I20260810 07:15:31.425091 24015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } }
I20260810 07:15:31.425266 24015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:31.425328 24015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d88e98d411d41038f21fc4875501737, State: Initialized, Role: FOLLOWER
I20260810 07:15:31.425734 24015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } }
I20260810 07:15:31.426158 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } }
I20260810 07:15:31.426268 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:31.426800 24015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:15:31.431385 24041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:31.431463 24041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:31.433485 23889 master_runner.cc:428] Detected that this master 3d88e98d411d41038f21fc4875501737 is joining an existing cluster
I20260810 07:15:31.433534 23889 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.117.189:43029
I20260810 07:15:31.443737 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.435509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41138)
0810 07:15:31.435761 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.435774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.435869 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.436169 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.436171 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.436550 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.436670 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.436676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437641 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.437645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.438552 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.438563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.439069 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.439073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.439188 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.439799 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.439814 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.440267 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.442097 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.442103 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.442435 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.442667 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.442671 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.442770 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.443157 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.443161 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.443253 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.443392 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.443528 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":39,"spinlock_wait_cycles":2432,"thread_start_us":99,"threads_started":1}
I20260810 07:15:31.443750 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.435405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.12.117.189:43029)
0810 07:15:31.435493 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.435497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.435840 (+ 343us) server_negotiation.cc:407] Connection header received
0810 07:15:31.435945 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.435948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.435983 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.436044 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.436772 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437528 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.438657 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.438954 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.439040 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.440407 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.440411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.440414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.440435 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.441984 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442521 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.442523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.442525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.442591 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442859 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.442863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.442864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.443066 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.443159 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.443441 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.443593 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":28}
I20260810 07:15:31.443776 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.435406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.12.117.190:41229)
0810 07:15:31.435526 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.435530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.435951 (+ 421us) server_negotiation.cc:407] Connection header received
0810 07:15:31.436127 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.436130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.436203 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.436257 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.436984 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437722 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.438544 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.438863 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.438954 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.440597 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.440601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.440604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.440631 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.442082 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442482 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.442485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.442487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.442580 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442900 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.442903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.442905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.443088 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.443155 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.443541 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.443645 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:31.444234 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.435524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45590)
0810 07:15:31.435867 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.435880 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.436013 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.436402 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.436405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.436684 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.436848 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.436856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.437810 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.437813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.438393 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.438401 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.439072 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.439076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.439192 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.440098 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.440110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.440444 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.442156 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.442161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.442394 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.442686 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.442689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.442691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.442796 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.443222 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.443225 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.443350 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.443938 (+ 588us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.444084 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":25,"thread_start_us":104,"threads_started":1}
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.190@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:31.456852 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.448415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41150)
0810 07:15:31.448742 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.448789 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.448916 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.449194 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.449198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.449511 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.449608 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.449614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.450590 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.450594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.451198 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.451207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.451762 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.451764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.451842 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.452302 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.452313 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.453818 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.455479 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.455489 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.455493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.455708 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.455965 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.455967 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.455968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.456037 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.456387 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.456391 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.456487 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.456593 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.456731 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":92,"threads_started":1}
I20260810 07:15:31.456910 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.448463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.12.117.189:43029)
0810 07:15:31.448564 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.448568 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.448859 (+ 291us) server_negotiation.cc:407] Connection header received
0810 07:15:31.449033 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.449036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.449083 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.449167 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.449705 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.450454 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.451324 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.451666 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.451766 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.453949 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.453953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.453956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.453988 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.455380 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.455792 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.455795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.455796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.455877 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.456108 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.456111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.456112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.456303 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.456393 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.456658 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.456773 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:31.458061 23097 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.117.189:43029: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "3d88e98d411d41038f21fc4875501737"
member_type: NON_VOTER
last_known_addr {
host: "127.12.117.189"
port: 43029
}
attrs {
promote: true
}
}
dest_uuid: "f8ae3d13348644b983abe5432ea684dc"
cas_config_opid_index: -1
I20260810 07:15:31.458688 23097 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } }
I20260810 07:15:31.460314 24053 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3d88e98d411d41038f21fc4875501737 (127.12.117.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.461208 23961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:31.462409 24053 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.117.189:43029
I20260810 07:15:31.462568 23961 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 07:15:31.462805 23889 master.cc:537] Master@127.12.117.189:43029 shutting down...
I20260810 07:15:31.463140 24053 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.463254 24053 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:31.464495 24060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:15:31.466753 23076 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260810 07:15:31.467818 23889 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:31.468096 23889 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:31.468199 23889 tablet_replica.cc:330] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: stopping tablet replica
WARNING: no policy specified for oryx/127.12.117.188@KRBTEST.COM; defaulting to no policy
W20260810 07:15:31.470938 23076 consensus_peers.cc:487] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc -> Peer 3d88e98d411d41038f21fc4875501737 (127.12.117.189:43029): Couldn't send request to peer 3d88e98d411d41038f21fc4875501737. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.188@KRBTEST.COM" created.
I20260810 07:15:31.483028 23889 master.cc:552] Master@127.12.117.189:43029 shutdown complete.
I20260810 07:15:31.483079 23889 master_runner.cc:305] Clearing existing system tablet
I20260810 07:15:31.483388 23889 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.483449 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.484516 23889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:31.485635 24068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.485831 23889 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.485942 23889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:31.485980 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "3d88e98d411d41038f21fc4875501737"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f305eacd7ab869664ea52156db1c673e"
server_key_iv: "7776a82d151f636eadfa9d26ffe5c065"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.188.keytab.
Entry for principal oryx/127.12.117.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.188.keytab.
I20260810 07:15:31.488101 23889 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:31.488363 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:45423
--webserver_interface=127.12.117.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.188.keytab
--principal=oryx/127.12.117.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:31.490319 23889 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:15:31.490408 23889 log.cc:1196] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 07:15:31.490672 23889 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Deleting consensus metadata
I20260810 07:15:31.490828 23889 master_runner.cc:315] Copying system tablet from 127.12.117.190:41229
I20260810 07:15:31.492184 23889 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: tablet copy: Beginning tablet copy session from remote peer at address 127.12.117.190:41229
I20260810 07:15:31.501796 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.492913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.12.117.190:41229)
0810 07:15:31.493040 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.493044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.493627 (+ 583us) server_negotiation.cc:407] Connection header received
0810 07:15:31.493827 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.493831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.493884 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.493937 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.494608 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.495360 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.496385 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.496711 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.496792 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.498271 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.498276 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.498279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.498305 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.500115 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.500627 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.500631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.500633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.500710 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.500970 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.500974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.500975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.501150 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.501238 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.501572 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.501680 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:15:31.502274 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.493134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45602)
0810 07:15:31.493532 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.493547 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.493710 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.494093 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.494096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.494305 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.494462 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.494470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.495491 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.495493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.496209 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.496221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.496954 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.496958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.497056 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.497611 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.497626 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.498110 (+ 484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.500271 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.500274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.500277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.500537 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.500798 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.500801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.500803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.500883 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.501243 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.501246 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.501367 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.501989 (+ 622us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.502135 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":111,"threads_started":1}
I20260810 07:15:31.502830 23128 tablet_copy_service.cc:140] P f8ae3d13348644b983abe5432ea684dc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 3d88e98d411d41038f21fc4875501737 ({username='oryx', principal='oryx/127.12.117.189@KRBTEST.COM'} at 127.0.0.1:45602)
I20260810 07:15:31.503000 23128 tablet_copy_service.cc:161] P f8ae3d13348644b983abe5432ea684dc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 3d88e98d411d41038f21fc4875501737 at {username='oryx', principal='oryx/127.12.117.189@KRBTEST.COM'} at 127.0.0.1:45602: session id = 3d88e98d411d41038f21fc4875501737-00000000000000000000000000000000
I20260810 07:15:31.504590 23128 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:15:31.506119 23889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:15:31.508153 23889 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: tablet copy: Starting download of 0 data blocks...
I20260810 07:15:31.508337 23889 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: tablet copy: Starting download of 1 WAL segments...
I20260810 07:15:31.510049 23889 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:15:31.511652 23128 tablet_copy_service.cc:342] P f8ae3d13348644b983abe5432ea684dc: Request end of tablet copy session 3d88e98d411d41038f21fc4875501737-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.117.189@KRBTEST.COM'} at 127.0.0.1:45602
I20260810 07:15:31.511745 23128 tablet_copy_service.cc:433] P f8ae3d13348644b983abe5432ea684dc: ending tablet copy session 3d88e98d411d41038f21fc4875501737-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 3d88e98d411d41038f21fc4875501737
I20260810 07:15:31.522394 23889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.524019 24089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.524464 24085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.525072 23889 server_base.cc:1034] running on GCE node
W20260810 07:15:31.527859 24082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.528045 23889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.528295 23889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.529438 23889 hybrid_clock.cc:648] HybridClock initialized: now 1786346131529423 us; error 28 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.530831 23889 init.cc:377] Logged in from keytab as oryx/127.12.117.189@KRBTEST.COM (short username oryx)
I20260810 07:15:31.531252 23889 webserver.cc:460] Webserver started at http://127.12.117.189:37559/ using document root <none> and password file <none>
I20260810 07:15:31.531512 23889 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.531548 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.532619 23889 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:31.533466 24110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.533679 23889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:31.533747 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "3d88e98d411d41038f21fc4875501737"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f305eacd7ab869664ea52156db1c673e"
server_key_iv: "7776a82d151f636eadfa9d26ffe5c065"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.533840 23889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.570288 23889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.572082 23889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.576759 23889 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.189:43029
I20260810 07:15:31.576779 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.189:43029 every 8 connection(s)
I20260810 07:15:31.577309 23889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
I20260810 07:15:31.578691 24205 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:31.579546 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Bootstrap starting.
I20260810 07:15:31.587167 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:31.587423 24205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: Bootstrap complete.
I20260810 07:15:31.587901 24205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } }
I20260810 07:15:31.588012 24205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 3d88e98d411d41038f21fc4875501737, State: Initialized, Role: LEARNER
I20260810 07:15:31.588112 24205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } }
I20260810 07:15:31.588409 24208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.588534 24205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:31.588594 24208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:31.592105 24220 catalog_manager.cc:1259] Loaded cluster ID: 80aca1c9dc494a7ebde5fc8c678672eb
I20260810 07:15:31.592155 24220 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: loading cluster ID for follower catalog manager: success
I20260810 07:15:31.593626 24220 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: acquiring CA information for follower catalog manager: success
I20260810 07:15:31.594671 24220 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 07:15:31.635517 24070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:31.635772 24070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:31.635810 24070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:31.639809 24070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:31.639858 24070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:31.639876 24070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:31.639891 24070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:31.639909 24070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:31.644940 24070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.188.keytab
--principal=oryx/127.12.117.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:45423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.117.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:31.645810 24070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:31.646697 24070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.654764 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.654770 24227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.654770 24230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.655007 24070 server_base.cc:1034] running on GCE node
I20260810 07:15:31.655555 24070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.656555 24070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.657719 24070 hybrid_clock.cc:648] HybridClock initialized: now 1786346131657701 us; error 31 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.660360 24070 init.cc:377] Logged in from keytab as oryx/127.12.117.188@KRBTEST.COM (short username oryx)
I20260810 07:15:31.661224 24070 webserver.cc:460] Webserver started at http://127.12.117.188:44285/ using document root <none> and password file <none>
I20260810 07:15:31.661736 24070 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.661803 24070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.662003 24070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:31.664014 24070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/instance:
uuid: "f17cb3ec531249839cb94887948a331c"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f589f250a6c0cdd351b521a9ee792234"
server_key_iv: "7b3e0417e12fa2c507888e6191c4fc0c"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.664559 24070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal/instance:
uuid: "f17cb3ec531249839cb94887948a331c"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f589f250a6c0cdd351b521a9ee792234"
server_key_iv: "7b3e0417e12fa2c507888e6191c4fc0c"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.668121 24070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:31.670435 24252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.671665 24070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:31.671782 24070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "f17cb3ec531249839cb94887948a331c"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f589f250a6c0cdd351b521a9ee792234"
server_key_iv: "7b3e0417e12fa2c507888e6191c4fc0c"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.671876 24070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.685096 24070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.687624 24070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:31.687786 24070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.694401 24070 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.188:45423
I20260810 07:15:31.694657 24350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.188:45423 every 8 connection(s)
I20260810 07:15:31.695344 24070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
I20260810 07:15:31.696277 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 24070
I20260810 07:15:31.698675 24354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:31.702389 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.188@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:31.710427 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.697808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.12.117.188:45423)
0810 07:15:31.698232 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.698246 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.698280 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:15:31.698965 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.698982 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.699354 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.699668 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.700495 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.701504 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.702354 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.702847 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.702993 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.705663 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.705670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.705681 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.705712 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.707415 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.707843 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.707849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.707853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.707978 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.708423 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.708427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.708429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.708789 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.708992 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.709263 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.709531 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"spinlock_wait_cycles":1920,"thread_start_us":134,"threads_started":1}
I20260810 07:15:31.713420 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.703928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45616)
0810 07:15:31.704378 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.704424 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.704545 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.704835 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.704839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.705205 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.705374 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.705382 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.706367 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.706369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.707099 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.707109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.707696 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.707700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.707816 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.708287 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.708299 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.709603 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.711909 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.711916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.711920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.712194 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.712494 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.712496 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.712498 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.712582 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.712953 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.712957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.713053 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.713157 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.713286 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"spinlock_wait_cycles":3328,"thread_start_us":90,"threads_started":1}
I20260810 07:15:31.713418 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.704102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45616 (local address 127.12.117.190:41229)
0810 07:15:31.704249 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.704254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.704499 (+ 245us) server_negotiation.cc:407] Connection header received
0810 07:15:31.704651 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.704655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.704707 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.704797 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.705505 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.706248 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.707226 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.707583 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.707668 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.709743 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.709747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.709750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.709780 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.711804 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.712305 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.712309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.712311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.712408 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.712664 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.712667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.712669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.712875 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.712953 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.713201 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.713293 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:15:31.714372 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.188@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:31.721271 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.714786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41156)
0810 07:15:31.714911 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.714926 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.714998 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.715347 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.715349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.715472 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.715570 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.715577 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.716371 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.716373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.716897 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.716903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.717287 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.717289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.717352 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.717681 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.717691 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.718742 (+ 1051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.720192 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.720194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.720196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.720374 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.720587 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.720588 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.720590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.720655 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.720929 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.720931 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.721000 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.721068 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.721168 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 07:15:31.721312 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.714888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.12.117.189:43029)
0810 07:15:31.715138 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.715142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.715153 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:31.715190 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.715192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.715229 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.715331 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.715663 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.716295 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.716982 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.717209 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.717283 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.718880 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.718882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.718884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.718908 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.720100 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.720449 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.720451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.720452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.720528 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.720712 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.720714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.720715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.720868 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.720936 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.721108 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.721199 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":109,"threads_started":1}
I20260810 07:15:31.722014 24354 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.188@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
I20260810 07:15:31.728647 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.722427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.0.0.1:46378)
0810 07:15:31.722524 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.722537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.722594 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.722769 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.722771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.722885 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.722951 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.722955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.723959 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.723961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.724434 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.724440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.724804 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.724806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.724858 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.725189 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.725199 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.726212 (+ 1013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.727618 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.727621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.727622 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.727793 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.727991 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.727993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.727994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.728056 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.728316 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.728318 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.728384 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.728459 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.728544 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 07:15:31.728725 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.722482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.12.117.188:45423)
0810 07:15:31.722582 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.722585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.722596 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:31.722667 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.722669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.722700 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.722751 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.723032 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.723872 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.724525 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.724735 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.724797 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.726348 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.726351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.726353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.726385 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.727527 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.727864 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.727867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.727869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.727932 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.728113 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.728114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.728115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.728254 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.728320 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.728516 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.728601 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:31.731575 24354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Bootstrap starting.
I20260810 07:15:31.733769 24354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:31.734453 24354 log.cc:824] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:31.736302 24354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: No bootstrap required, opened a new log
I20260810 07:15:31.739303 24354 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } }
I20260810 07:15:31.739531 24354 raft_consensus.cc:374] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:31.739575 24354 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17cb3ec531249839cb94887948a331c, State: Initialized, Role: FOLLOWER
I20260810 07:15:31.739960 24354 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } }
I20260810 07:15:31.740388 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } } }
I20260810 07:15:31.740509 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:31.741020 24354 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:15:31.745306 24386 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:31.745379 24386 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:31.748627 24070 master_runner.cc:428] Detected that this master f17cb3ec531249839cb94887948a331c is joining an existing cluster
I20260810 07:15:31.748688 24070 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.117.188:45423
I20260810 07:15:31.760332 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.750899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45630)
0810 07:15:31.751611 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.751626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.751724 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.752119 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.752122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.752381 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.752596 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.752604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753667 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.753670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.755212 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.755223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755942 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.756032 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.756382 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.756396 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.756814 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.758671 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.758674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.758676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.758915 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.759329 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.759332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.759334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.759428 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.759846 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.759848 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.759938 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.760028 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.760155 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":38,"thread_start_us":115,"threads_started":1}
I20260810 07:15:31.760493 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.751433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45630 (local address 127.12.117.190:41229)
0810 07:15:31.751638 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.751642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.751683 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:15:31.751853 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.751856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.751906 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.751991 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.752736 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753530 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755385 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755727 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755799 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.756970 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.756975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.756978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.757005 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.758559 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.759017 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.759021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.759023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.759125 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.759535 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.759539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.759540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.759756 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.759852 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.760102 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.760231 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260810 07:15:31.763547 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 23196
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346130, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.190@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
I20260810 07:15:31.777289 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:38829
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:41229
--webserver_port=42961
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:31.784859 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.750995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.12.117.189:43029)
0810 07:15:31.751102 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.751106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.751794 (+ 688us) server_negotiation.cc:407] Connection header received
0810 07:15:31.751912 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.751915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.751965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.752013 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.752747 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753505 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.754768 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755110 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755174 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.780777 (+ 25603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.780781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.780784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.780813 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.782407 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.783360 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.783365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.783367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.783467 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.783832 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.783836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.783838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.784077 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.784166 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.784539 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.784696 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:31.785379 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.750904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41160)
0810 07:15:31.751714 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.751725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.751802 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.752145 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.752148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.752451 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.752616 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.752624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753640 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.753643 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.754638 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.754649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755350 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.755437 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.755959 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.755972 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.780027 (+ 24055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.782552 (+ 2525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.782554 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.782557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.782785 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.783597 (+ 812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.783601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.783603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.783715 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.784231 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.784234 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.784335 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.785051 (+ 716us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.785197 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":77,"thread_start_us":75,"threads_started":1}
I20260810 07:15:31.786631 24397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.751381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.0.0.1:46386)
0810 07:15:31.751907 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.751918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.752009 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.752285 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.752287 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.752487 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.752629 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.752635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753628 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.753631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.754638 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.754649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755559 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.755646 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.779729 (+ 24083us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.779743 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.780121 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.784502 (+ 4381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.784504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.784506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.784721 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.785064 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.785067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.785069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.785152 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.786139 (+ 987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.786142 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.786232 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.786329 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.786462 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":92,"thread_start_us":86,"threads_started":1}
I20260810 07:15:31.787065 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.751434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46386 (local address 127.12.117.188:45423)
0810 07:15:31.751541 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.751545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.751968 (+ 423us) server_negotiation.cc:407] Connection header received
0810 07:15:31.752083 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.752086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.752132 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.752174 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.752729 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.753505 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.754760 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.755067 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.755146 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.780258 (+ 25112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.780261 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.780263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.780284 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.784382 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.784807 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.784809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.784811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.784878 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.785236 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.785239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.785241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.785422 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.785529 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.786822 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.786924 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43,"spinlock_wait_cycles":896}
I20260810 07:15:31.790236 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.770083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.0.0.1:46390)
0810 07:15:31.770207 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.770222 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.770302 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.771715 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.771719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.771904 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.772005 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.772011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.773122 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.773125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.773916 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.773928 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.774547 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.774550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.774676 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.775121 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.775136 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.776730 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.788534 (+ 11804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.788538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.788540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.788814 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.789115 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.789119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.789121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.789214 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.789706 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.789709 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.789812 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.789938 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.790081 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:15:31.790725 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.770140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.12.117.188:45423)
0810 07:15:31.771426 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.771437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.771459 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:15:31.771521 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.771524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.771582 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.772112 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.772209 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.772995 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.774035 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.774402 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.774542 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.776891 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.776895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.776898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.776922 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.788407 (+ 11485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.788920 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.788923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.788925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.789022 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.789303 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.789306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.789308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.789514 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.789706 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.790446 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.790559 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1222,"thread_start_us":86,"threads_started":1}
I20260810 07:15:31.791608 23097 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.117.188:45423: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f17cb3ec531249839cb94887948a331c"
member_type: NON_VOTER
last_known_addr {
host: "127.12.117.188"
port: 45423
}
attrs {
promote: true
}
}
dest_uuid: "f8ae3d13348644b983abe5432ea684dc"
cas_config_opid_index: 5
I20260810 07:15:31.792199 23097 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.793463 24055 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f17cb3ec531249839cb94887948a331c (127.12.117.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.794875 24055 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.117.188:45423
I20260810 07:15:31.795037 24055 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.795128 24055 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:31.796252 24307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:31.797557 24307 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:15:31.798091 24053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:31.798465 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.793615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41174)
0810 07:15:31.794001 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.794015 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.794090 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.794444 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.794447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.794463 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:31.794542 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.794547 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.795663 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.795666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.796938 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.796949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.798067 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.798071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.798163 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.798215 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.798284 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.798349 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322}
I20260810 07:15:31.798851 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.793823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41174 (local address 127.12.117.189:43029)
0810 07:15:31.793924 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.793928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.794160 (+ 232us) server_negotiation.cc:407] Connection header received
0810 07:15:31.794204 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.794207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.794260 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.794335 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:31.794681 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.795524 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.797079 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.797734 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.797863 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.798004 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.798663 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.798740 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:31.799371 24153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 LEARNER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 07:15:31.799767 24053 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 07:15:31.801468 24153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f17cb3ec531249839cb94887948a331c (127.12.117.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.801796 24070 master.cc:537] Master@127.12.117.188:45423 shutting down...
I20260810 07:15:31.803174 24307 raft_consensus.cc:858] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.803210 24208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ae3d13348644b983abe5432ea684dc. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.803334 24208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: LEARNER
W20260810 07:15:31.803256 24307 raft_consensus.cc:1565] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.803406 24208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.803469 24208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:31.804711 24307 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 3d88e98d411d41038f21fc4875501737 (127.12.117.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.805568 24053 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: attempting to promote NON_VOTER 3d88e98d411d41038f21fc4875501737 to VOTER
I20260810 07:15:31.805918 24053 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.806756 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ae3d13348644b983abe5432ea684dc. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.806851 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:31.806947 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.807016 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:31.807116 24365 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: NON_VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: true } } }
I20260810 07:15:31.807186 24365 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 07:15:31.807794 24153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 LEARNER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260810 07:15:31.807996 23072 consensus_peers.cc:487] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc -> Peer f17cb3ec531249839cb94887948a331c (127.12.117.188:45423): Couldn't send request to peer f17cb3ec531249839cb94887948a331c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:31.808182 24055 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 07:15:31.809746 24055 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3d88e98d411d41038f21fc4875501737 (127.12.117.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.810994 24055 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.811084 24055 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:31.811187 24055 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.811939 24153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 3d88e98d411d41038f21fc4875501737 (127.12.117.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.811249 24055 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:31.813671 24406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.813817 24406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:31.813848 24404 mvcc.cc:205] Tried to move back new op lower bound from 7316873750802444288 to 7316873750649724928. Current Snapshot: MvccSnapshot[applied={T|T < 7316873750682476544}]
I20260810 07:15:31.816423 24070 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 07:15:31.816644 24070 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 07:15:31.816715 24070 tablet_replica.cc:330] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: stopping tablet replica
I20260810 07:15:31.835626 24070 master.cc:552] Master@127.12.117.188:45423 shutdown complete.
I20260810 07:15:31.835680 24070 master_runner.cc:305] Clearing existing system tablet
I20260810 07:15:31.835932 24070 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.835987 24070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.837258 24070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 07:15:31.838122 24418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.838339 24070 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.838472 24070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:31.838510 24070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "f17cb3ec531249839cb94887948a331c"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f589f250a6c0cdd351b521a9ee792234"
server_key_iv: "7b3e0417e12fa2c507888e6191c4fc0c"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.840180 24070 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:31.842420 24070 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 07:15:31.842519 24070 log.cc:1196] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 07:15:31.842772 24070 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Deleting consensus metadata
I20260810 07:15:31.842919 24070 master_runner.cc:315] Copying system tablet from 127.12.117.190:41229
I20260810 07:15:31.844009 24070 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: tablet copy: Beginning tablet copy session from remote peer at address 127.12.117.190:41229
I20260810 07:15:31.852669 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.844574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.12.117.190:41229)
0810 07:15:31.844688 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.844692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.844873 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:15:31.844991 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.844994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.845044 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.845131 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.845611 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.846412 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.847593 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.847939 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.848063 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.849327 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.849331 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.849334 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.849361 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.850969 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.851466 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.851469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.851471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.851552 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:31.851815 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:31.851818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:31.851819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:31.852012 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:31.852076 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:31.852388 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 07:15:31.852511 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:31.853144 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.844486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:45642)
0810 07:15:31.844782 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.844796 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.844909 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.845160 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.845163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.845381 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.845496 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.845503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.846537 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.846540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.847456 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.847468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.848068 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.848072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.848167 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.848716 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.848731 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.849149 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.851089 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.851091 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.851094 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.851352 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.851651 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.851654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.851656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.851730 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:31.852141 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:31.852144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:31.852233 (+ 89us) client_negotiation.cc:769] Sending connection context
0810 07:15:31.852839 (+ 606us) client_negotiation.cc:240] Negotiation successful
0810 07:15:31.852976 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
I20260810 07:15:31.853631 23128 tablet_copy_service.cc:140] P f8ae3d13348644b983abe5432ea684dc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f17cb3ec531249839cb94887948a331c ({username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:45642)
I20260810 07:15:31.853727 23128 tablet_copy_service.cc:161] P f8ae3d13348644b983abe5432ea684dc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f17cb3ec531249839cb94887948a331c at {username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:45642: session id = f17cb3ec531249839cb94887948a331c-00000000000000000000000000000000
I20260810 07:15:31.854759 23128 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 07:15:31.855924 24070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 07:15:31.857972 24070 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: tablet copy: Starting download of 0 data blocks...
I20260810 07:15:31.858150 24070 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: tablet copy: Starting download of 1 WAL segments...
I20260810 07:15:31.859593 24070 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 07:15:31.861128 23128 tablet_copy_service.cc:342] P f8ae3d13348644b983abe5432ea684dc: Request end of tablet copy session f17cb3ec531249839cb94887948a331c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:45642
I20260810 07:15:31.861208 23128 tablet_copy_service.cc:433] P f8ae3d13348644b983abe5432ea684dc: ending tablet copy session f17cb3ec531249839cb94887948a331c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f17cb3ec531249839cb94887948a331c
I20260810 07:15:31.868556 24070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.870262 24432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.870368 24433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.870604 24435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.871031 24070 server_base.cc:1034] running on GCE node
I20260810 07:15:31.871227 24070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.871546 24070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.872704 24070 hybrid_clock.cc:648] HybridClock initialized: now 1786346131872690 us; error 27 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.874493 24070 init.cc:377] Logged in from keytab as oryx/127.12.117.188@KRBTEST.COM (short username oryx)
I20260810 07:15:31.875063 24070 webserver.cc:460] Webserver started at http://127.12.117.188:34743/ using document root <none> and password file <none>
I20260810 07:15:31.875331 24070 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.875432 24070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.876742 24070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:31.877789 24447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.878015 24070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:31.878084 24070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "f17cb3ec531249839cb94887948a331c"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "f589f250a6c0cdd351b521a9ee792234"
server_key_iv: "7b3e0417e12fa2c507888e6191c4fc0c"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.878168 24070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.889113 24070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.891227 24070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.896085 24070 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.188:45423
I20260810 07:15:31.896116 24547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.188:45423 every 8 connection(s)
I20260810 07:15:31.896622 24070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
I20260810 07:15:31.898454 24549 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:31.899530 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Bootstrap starting.
I20260810 07:15:31.907356 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:31.907630 24549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Bootstrap complete.
I20260810 07:15:31.908154 24549 raft_consensus.cc:348] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.908298 24549 raft_consensus.cc:729] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f17cb3ec531249839cb94887948a331c, State: Initialized, Role: LEARNER
I20260810 07:15:31.908413 24549 consensus_queue.cc:260] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } }
I20260810 07:15:31.908726 24556 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:31.908891 24549 sys_catalog.cc:564] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:31.908915 24556 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: LEARNER
I20260810 07:15:31.912290 24574 catalog_manager.cc:1259] Loaded cluster ID: 80aca1c9dc494a7ebde5fc8c678672eb
I20260810 07:15:31.912348 24574 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: loading cluster ID for follower catalog manager: success
I20260810 07:15:31.914364 24574 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: acquiring CA information for follower catalog manager: success
W20260810 07:15:31.914572 24401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:31.914877 24401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:31.914955 24401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 07:15:31.915781 24574 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 07:15:31.919235 24401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:31.919312 24401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:31.919404 24401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:31.924558 24401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:38829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=42961
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:31.925467 24401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:31.926375 24401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:31.935670 24577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.935678 24576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:31.935729 24579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:31.935854 24401 server_base.cc:1034] running on GCE node
I20260810 07:15:31.936297 24401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:31.936903 24401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:31.938082 24401 hybrid_clock.cc:648] HybridClock initialized: now 1786346131938066 us; error 26 us; skew 500 ppm
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:31.940670 24401 init.cc:377] Logged in from keytab as oryx/127.12.117.129@KRBTEST.COM (short username oryx)
I20260810 07:15:31.941498 24401 webserver.cc:460] Webserver started at http://127.12.117.129:42961/ using document root <none> and password file <none>
I20260810 07:15:31.942030 24401 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:31.942123 24401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:31.945855 24401 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:31.947815 24589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.948978 24401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:31.949086 24401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "801f9248a6ea48829543c852775bbd1b"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "2bfce837b824e4cff95a4259d20fff72"
server_key_iv: "3f7aadc1f0af6aa993945fc8a7b21ed4"
server_key_version: "encryptionkey@0"
I20260810 07:15:31.949590 24401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:31.965001 24401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:31.967690 24401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:31.967856 24401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:31.968395 24401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:31.969211 24401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:31.969260 24401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.969312 24401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:31.969342 24401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:31.971460 23635 heartbeater.cc:503] Master 127.12.117.190:41229 was elected leader, sending a full tablet report...
I20260810 07:15:31.978302 24401 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38829
I20260810 07:15:31.978346 24788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38829 every 8 connection(s)
I20260810 07:15:31.979216 24401 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:31.984170 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 24401
I20260810 07:15:31.984366 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 23421
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
Aug 10 07:15:31 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
I20260810 07:15:32.002846 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.980835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:38453 (local address 127.12.117.190:41229)
0810 07:15:31.981022 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.981027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.982362 (+ 1335us) server_negotiation.cc:407] Connection header received
0810 07:15:31.983340 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.983343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.983388 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.983456 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.989175 (+ 5719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.990286 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.992517 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.992925 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.993012 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.997011 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:31.997016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:31.997019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:31.997051 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:31.999305 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.000135 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.000139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.000142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.000256 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.001283 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.001287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.001289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.001558 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.001673 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.002504 (+ 831us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.002662 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 07:15:32.003005 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.981482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:56639 (local address 127.12.117.188:45423)
0810 07:15:31.981717 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.981721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.982598 (+ 877us) server_negotiation.cc:407] Connection header received
0810 07:15:31.983353 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.983356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.983410 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.983506 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:31.995962 (+ 12456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.996807 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.997717 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.997998 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.998058 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.000056 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.000060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.000063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.000089 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.001514 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.001909 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.001913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.001914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.001991 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.002221 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.002224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.002226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.002403 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.002479 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.002756 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.002864 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":75,"threads_started":1}
I20260810 07:15:32.004424 24802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.981275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.12.117.129:56639)
0810 07:15:31.982516 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.982526 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.982838 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.983637 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.983640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.983899 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.995829 (+ 11930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.995840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.996923 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.996925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.997605 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.997613 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.998189 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.998192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.998268 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.998669 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.998681 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.999919 (+ 1238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.001618 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.001621 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.001623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.001808 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.002081 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.002083 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.002085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.002150 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.002541 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.002544 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.002622 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.003161 (+ 539us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.003507 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1177,"mutex_wait_us":214,"spinlock_wait_cycles":17024,"thread_start_us":99,"threads_started":1}
I20260810 07:15:32.004429 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.981275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:38453)
0810 07:15:31.982269 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.982280 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.982835 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.983504 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.983506 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.983897 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:31.988971 (+ 5074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.989000 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.990544 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.990548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.992360 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:31.992373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:31.993234 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:31.993240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:31.993485 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:31.994551 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:31.994574 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:31.996793 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:31.999497 (+ 2704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:31.999504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:31.999522 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:31.999997 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.000672 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.000675 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.000677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.001148 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.001755 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.001763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.002159 (+ 396us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.003096 (+ 937us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.003622 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":150,"spinlock_wait_cycles":21760,"thread_start_us":144,"threads_started":1}
I20260810 07:15:32.005656 24791 heartbeater.cc:347] Connected to a master server at 127.12.117.188:45423
I20260810 07:15:32.005898 24791 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.006361 24791 heartbeater.cc:511] Master 127.12.117.188:45423 requested a full tablet report, sending...
I20260810 07:15:32.005674 24792 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:32.006841 24792 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.006986 24792 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:32.007771 23097 ts_manager.cc:194] Re-registered known tserver with Master: 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:32.008272 24472 ts_manager.cc:194] Registered new tserver with Master: 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:32.008432 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:38453
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346131, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:32.018834 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.980847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:33447 (local address 127.12.117.189:43029)
0810 07:15:31.981122 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:31.981126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:31.982136 (+ 1010us) server_negotiation.cc:407] Connection header received
0810 07:15:31.983049 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:31.983052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:31.983117 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:31.983189 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.012233 (+ 29044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.012746 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.013538 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.013761 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.013851 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.015784 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.015788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.015790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.015815 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.017333 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.017742 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.017745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.017747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.017835 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.018052 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.018055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.018057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.018265 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.018366 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.018571 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.018697 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260810 07:15:32.019004 24796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:31.981274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.12.117.129:33447)
0810 07:15:31.981990 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:31.982041 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:31.982831 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:31.983394 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:31.983403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:31.983920 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.012093 (+ 28173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.012104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.012855 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.012858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.013420 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.013428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.013874 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.013878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.013980 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.014397 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.014409 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.015638 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.017440 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.017443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.017444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.017652 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.017911 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.017912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.017914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.017983 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.018343 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.018345 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.018416 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.018492 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.018618 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":23,"spinlock_wait_cycles":11648,"thread_start_us":151,"threads_started":1}
I20260810 07:15:32.019687 24793 heartbeater.cc:347] Connected to a master server at 127.12.117.189:43029
I20260810 07:15:32.019780 24793 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.019960 24793 heartbeater.cc:511] Master 127.12.117.189:43029 requested a full tablet report, sending...
I20260810 07:15:32.020954 24124 ts_manager.cc:194] Registered new tserver with Master: 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:32.022773 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:45751
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:41229
--webserver_port=43217
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:32.131067 24806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:32.131292 24806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:32.131330 24806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:32.135424 24806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:32.135488 24806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:32.135551 24806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:32.140954 24806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:45751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=43217
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:32.141906 24806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:32.142823 24806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:32.151791 24817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:32.151826 24822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:32.151824 24806 server_base.cc:1034] running on GCE node
W20260810 07:15:32.151846 24818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:32.152498 24806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:32.153067 24806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:32.154247 24806 hybrid_clock.cc:648] HybridClock initialized: now 1786346132154216 us; error 39 us; skew 500 ppm
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:32.156937 24806 init.cc:377] Logged in from keytab as oryx/127.12.117.130@KRBTEST.COM (short username oryx)
I20260810 07:15:32.157742 24806 webserver.cc:460] Webserver started at http://127.12.117.130:43217/ using document root <none> and password file <none>
I20260810 07:15:32.158265 24806 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:32.158350 24806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:32.161963 24806 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:32.163846 24834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.164963 24806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:32.165068 24806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "490e07200bcc4287bfc4af6e1a9753e3"
format_stamp: "Formatted at 2026-08-10 07:15:30 on dist-test-slave-gjrz"
server_key: "d82359ddef4a240a3e0909318000bcbd"
server_key_iv: "030040d80cc93359c71688c74c3dabd4"
server_key_version: "encryptionkey@0"
I20260810 07:15:32.165557 24806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:32.186254 23880 heartbeater.cc:503] Master 127.12.117.190:41229 was elected leader, sending a full tablet report...
I20260810 07:15:32.187310 24806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:32.190047 24806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:32.190186 24806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:32.190675 24806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:32.191499 24806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:32.191546 24806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.191583 24806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:32.191603 24806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.199805 24806 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:45751
I20260810 07:15:32.199829 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:45751 every 8 connection(s)
I20260810 07:15:32.200682 24806 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:32.208886 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 24806
I20260810 07:15:32.209606 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 23646
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:32.218477 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:48577 (local address 127.12.117.188:45423)
0810 07:15:32.202735 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.202739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.203855 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 07:15:32.204421 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.204425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.204487 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.204563 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.205959 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.206773 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208061 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208417 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208548 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.214336 (+ 5788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.214341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.214345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.214377 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.216360 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216952 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.216955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.216957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.217047 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.217380 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.217383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.217385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.217590 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.217670 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.218195 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.218318 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:32.218478 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:38343 (local address 127.12.117.190:41229)
0810 07:15:32.202430 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.202435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.203975 (+ 1540us) server_negotiation.cc:407] Connection header received
0810 07:15:32.204447 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.204451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.204504 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.204555 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.205960 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.206494 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208114 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208447 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208521 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.214348 (+ 5827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.214352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.214353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.214378 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.216374 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216958 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.216962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.216963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.217047 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.217380 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.217383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.217385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.217590 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.217670 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.218195 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.218318 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:15:32.219904 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:57579 (local address 127.12.117.189:43029)
0810 07:15:32.203390 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.203395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.203413 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:15:32.204448 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.204451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.204504 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.204555 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.205991 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.206784 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208142 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208470 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208556 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.211948 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.211953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.211957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.211990 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.215688 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216418 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.216423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.216426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.216547 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.217027 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.217031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.217033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.217315 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.217399 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.219581 (+ 2182us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.219730 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372}
I20260810 07:15:32.220041 25037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.12.117.130:48577)
0810 07:15:32.203758 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.203769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.204216 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.204754 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.204761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.205256 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.205795 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.205810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.207340 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.207342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.207943 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.207951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208575 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.208774 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.209464 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.209487 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.214182 (+ 4695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.216491 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.216497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.216833 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.217157 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.217161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.217163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.217272 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.217730 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.217733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.217906 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.218720 (+ 814us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.219004 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":132,"spinlock_wait_cycles":15488,"thread_start_us":111,"threads_started":1}
I20260810 07:15:32.220041 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:38343)
0810 07:15:32.203884 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.203897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.204225 (+ 328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.204756 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.204764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.205263 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.205803 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.205813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.206636 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.206638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.207972 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.207983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208672 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.208821 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.212571 (+ 3750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.212599 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.214182 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.216491 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.216497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.216833 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.217157 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.217161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.217163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.217281 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.217730 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.217733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.217906 (+ 173us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.218645 (+ 739us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.219066 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"mutex_wait_us":491,"spinlock_wait_cycles":121984,"thread_start_us":97,"threads_started":1}
I20260810 07:15:32.221042 25034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.202766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.12.117.130:57579)
0810 07:15:32.203256 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.203327 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.204215 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.204756 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.204764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.205256 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.205808 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.205814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.206893 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.206897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.207973 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.207984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.208693 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.208697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.208885 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.209611 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.209626 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.211727 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.215840 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.215847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.215865 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.216279 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.216678 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.216681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.216684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.216890 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.217513 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.217521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.217916 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.220668 (+ 2752us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.220843 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":179,"spinlock_wait_cycles":16384,"thread_start_us":144,"threads_started":1}
I20260810 07:15:32.221240 25025 heartbeater.cc:347] Connected to a master server at 127.12.117.188:45423
I20260810 07:15:32.221567 25025 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.222278 25025 heartbeater.cc:511] Master 127.12.117.188:45423 requested a full tablet report, sending...
I20260810 07:15:32.221298 25027 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:32.222975 25027 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.223167 25027 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:32.223567 24472 ts_manager.cc:194] Registered new tserver with Master: 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751)
I20260810 07:15:32.223881 23097 ts_manager.cc:194] Re-registered known tserver with Master: 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751)
I20260810 07:15:32.224476 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:38343
I20260810 07:15:32.226007 25028 heartbeater.cc:347] Connected to a master server at 127.12.117.189:43029
I20260810 07:15:32.226076 25028 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.226212 25028 heartbeater.cc:511] Master 127.12.117.189:43029 requested a full tablet report, sending...
I20260810 07:15:32.226987 24124 ts_manager.cc:194] Registered new tserver with Master: 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751)
I20260810 07:15:32.239423 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:43867
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:41229
--webserver_port=38857
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:32.267143 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.263633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.0.0.1:46406)
0810 07:15:32.263756 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.263772 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.263850 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.264078 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.264092 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.264106 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.264185 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.264189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.265193 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.265196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.265934 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.265944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.266822 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.266825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.266899 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.266920 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.266974 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.267038 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:15:32.267189 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.263703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.12.117.188:45423)
0810 07:15:32.263844 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.263849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.263865 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:15:32.263917 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.263920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.263979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.264074 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.264283 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.265061 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.266051 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.266689 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.266807 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.266946 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.266999 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.267066 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:15:32.267725 24500 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 07:15:32.267969 24556 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8ae3d13348644b983abe5432ea684dc. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:32.268114 24556 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: LEARNER
W20260810 07:15:32.347910 25038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:32.348119 25038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:32.348157 25038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:32.351960 25038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:32.352005 25038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:32.352067 25038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:32.356976 25038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:43867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=38857
--tserver_master_addrs=127.12.117.190:41229,127.12.117.189:43029,127.12.117.188:45423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:32.357805 25038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:32.358691 25038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:32.367503 25046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:32.367496 25050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:32.367527 25048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:32.367745 25038 server_base.cc:1034] running on GCE node
I20260810 07:15:32.368104 25038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:32.368706 25038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:32.369853 25038 hybrid_clock.cc:648] HybridClock initialized: now 1786346132369822 us; error 39 us; skew 500 ppm
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:32.372447 25038 init.cc:377] Logged in from keytab as oryx/127.12.117.131@KRBTEST.COM (short username oryx)
I20260810 07:15:32.373200 25038 webserver.cc:460] Webserver started at http://127.12.117.131:38857/ using document root <none> and password file <none>
I20260810 07:15:32.373677 25038 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:32.373756 25038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:32.377254 25038 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260810 07:15:32.379208 25060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.380345 25038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:15:32.380457 25038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3"
format_stamp: "Formatted at 2026-08-10 07:15:31 on dist-test-slave-gjrz"
server_key: "a89d44b9b3d9664b5f29499d59e51c0a"
server_key_iv: "0c3ba8dff8da984a7833a4794cd744a9"
server_key_version: "encryptionkey@0"
I20260810 07:15:32.380920 25038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:32.402599 25038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:32.405102 25038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:32.405256 25038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:32.405754 25038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:32.406517 25038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:32.406563 25038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.406611 25038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:32.406637 25038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:32.414778 25038 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:43867
I20260810 07:15:32.414812 25254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:43867 every 8 connection(s)
I20260810 07:15:32.415676 25038 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.189@KRBTEST.COM
I20260810 07:15:32.425211 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 25038
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
Aug 10 07:15:32 dist-test-slave-gjrz krb5kdc[23012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346132, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.188@KRBTEST.COM
I20260810 07:15:32.429993 23167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.418631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:50017 (local address 127.12.117.190:41229)
0810 07:15:32.418736 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.418740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.418752 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:32.419512 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.419516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.419577 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.419624 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.420891 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.421668 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.422818 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423152 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423256 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.426620 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.426624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.426627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.426655 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.428245 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428718 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.428721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.428723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.428802 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.429104 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.429107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.429109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.429309 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.429382 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.429722 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.429839 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:32.430810 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.417818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.12.117.131:37715)
0810 07:15:32.418846 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.418857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.419285 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.419911 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.419913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.420254 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.420733 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.420743 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.421552 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.421554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.422767 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.422777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423440 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.423548 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.424045 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.424067 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.425712 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.427705 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.427711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.427725 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.428063 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.428380 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.428384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.428598 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.429099 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.429105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.429376 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.429562 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.429852 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":65,"spinlock_wait_cycles":32896,"thread_start_us":101,"threads_started":1}
I20260810 07:15:32.431062 25262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.417627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:50017)
0810 07:15:32.418311 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.418360 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.419257 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.419770 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.419777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.420247 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.420736 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.420743 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.421778 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.421781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.422669 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.422680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423323 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.423555 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.424174 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.424186 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.426413 (+ 2227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.428369 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428372 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.428374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.428614 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.428917 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428919 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.428921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.429003 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.429466 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.429469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.429548 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.430131 (+ 583us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.430261 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":138,"spinlock_wait_cycles":24064,"thread_start_us":133,"threads_started":1}
I20260810 07:15:32.431082 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.417267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35845 (local address 127.12.117.188:45423)
0810 07:15:32.417401 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.417406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.418691 (+ 1285us) server_negotiation.cc:407] Connection header received
0810 07:15:32.419477 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.419481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.419535 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.419609 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.420859 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.421668 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423034 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423293 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423389 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.427424 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.427428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.427431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.427459 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.429178 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.429719 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.429723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.429724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.429807 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.430100 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.430103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.430105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.430314 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.430379 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.430820 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.430933 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:15:32.431579 25264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.417627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.12.117.131:35845)
0810 07:15:32.418610 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.418623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.419279 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.419772 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.419779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.420231 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:32.420722 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.420737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.422127 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.422130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.422933 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.422940 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423520 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.423668 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.424310 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:32.424324 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:32.427252 (+ 2928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:32.429315 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.429318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.429320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.429609 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.429910 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:32.429913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:32.429915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:32.429995 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:32.430510 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:32.430513 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:32.430601 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.431250 (+ 649us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.431407 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":107,"spinlock_wait_cycles":30464,"thread_start_us":130,"threads_started":1}
I20260810 07:15:32.432196 25257 heartbeater.cc:347] Connected to a master server at 127.12.117.188:45423
I20260810 07:15:32.432353 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.417946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:37715 (local address 127.12.117.189:43029)
0810 07:15:32.418077 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.418082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.418936 (+ 854us) server_negotiation.cc:407] Connection header received
0810 07:15:32.419738 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.419742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.419794 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.419871 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.420882 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.421399 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.422882 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.423209 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.423326 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.425909 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.425914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.425917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.425959 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.427569 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428187 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.428191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.428193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.428281 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.428712 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.428714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.428716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.428957 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.429049 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.432101 (+ 3052us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.432219 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 07:15:32.432488 25258 heartbeater.cc:347] Connected to a master server at 127.12.117.190:41229
I20260810 07:15:32.432662 25258 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.432667 25257 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.433221 25258 heartbeater.cc:511] Master 127.12.117.190:41229 requested a full tablet report, sending...
I20260810 07:15:32.433221 25257 heartbeater.cc:511] Master 127.12.117.188:45423 requested a full tablet report, sending...
I20260810 07:15:32.434176 25259 heartbeater.cc:347] Connected to a master server at 127.12.117.189:43029
I20260810 07:15:32.434254 25259 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:32.434260 23097 ts_manager.cc:194] Re-registered known tserver with Master: bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:32.434345 24472 ts_manager.cc:194] Registered new tserver with Master: bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:32.434410 25259 heartbeater.cc:511] Master 127.12.117.189:43029 requested a full tablet report, sending...
I20260810 07:15:32.434945 23097 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:50017
I20260810 07:15:32.435063 24124 ts_manager.cc:194] Registered new tserver with Master: bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:32.439883 25272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.427154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45652 (local address 127.12.117.190:41229)
0810 07:15:32.427794 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.427798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.428164 (+ 366us) server_negotiation.cc:407] Connection header received
0810 07:15:32.428208 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.428211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.428260 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.428348 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.429347 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.430201 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.431574 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.432414 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.432536 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.436688 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.436692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.436694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.436719 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.438286 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.438688 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.438691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.438693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.438772 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.439038 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.439042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.439044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.439204 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.439341 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.439607 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.439724 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":73,"threads_started":1}
I20260810 07:15:32.444204 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.432443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.12.117.188:45423)
0810 07:15:32.432558 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.432562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.435898 (+ 3336us) server_negotiation.cc:407] Connection header received
0810 07:15:32.436077 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.436080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.436123 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.436170 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.437165 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.437727 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.438836 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.439150 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.439218 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.441115 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.441118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.441121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.441145 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.442439 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.442895 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.442898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.442900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.442985 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.443312 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.443315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.443317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.443528 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.443623 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.443959 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.444065 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:15:32.444209 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.434339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.12.117.189:43029)
0810 07:15:32.434444 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.434447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.435907 (+ 1460us) server_negotiation.cc:407] Connection header received
0810 07:15:32.436092 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.436095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.436132 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.436176 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.437142 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.437874 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.438907 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.439221 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.439315 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.441117 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.441121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.441123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.441150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.442727 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.443099 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.443103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.443104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.443180 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.443451 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.443455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.443456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.443624 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.443736 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.443967 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.444070 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:32.443006 23097 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:32.446514 23097 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:32.449051 25282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: NON_VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: true } } }
I20260810 07:15:32.449156 25282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:32.450630 25282 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: attempting to promote NON_VOTER f17cb3ec531249839cb94887948a331c to VOTER
I20260810 07:15:32.450939 25282 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:32.452035 24500 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 LEARNER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:15:32.452128 24153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Refusing update from remote peer f8ae3d13348644b983abe5432ea684dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 07:15:32.452415 25282 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 07:15:32.452585 25281 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 07:15:32.454195 25282 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f17cb3ec531249839cb94887948a331c (127.12.117.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.454591 24153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f17cb3ec531249839cb94887948a331c (127.12.117.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.455677 25282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.455785 25282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:32.455902 25282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.455979 25282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:32.456076 25282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.456139 25282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:32.456058 24500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f17cb3ec531249839cb94887948a331c (127.12.117.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.457274 24556 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.457361 24556 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:32.458563 25296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8ae3d13348644b983abe5432ea684dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:32.458724 25296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:32.465932 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.461099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:45751 (local address 127.0.0.1:54610)
0810 07:15:32.461557 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.461570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.461644 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.462336 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.462339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.462355 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.462469 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.462475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.463756 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.463759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.464608 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.464618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.465553 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.465557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.465659 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.465685 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.465729 (+ 44us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.465791 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":3}
I20260810 07:15:32.466629 25301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.461477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.12.117.130:45751)
0810 07:15:32.461795 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.461812 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.461835 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:15:32.461930 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.461936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.462170 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.462386 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.462565 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.463638 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.464739 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.465418 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.466134 (+ 716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.466278 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.466327 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.466489 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
I20260810 07:15:32.467579 25302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.461922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38829 (local address 127.0.0.1:46376)
0810 07:15:32.462165 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.462179 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.462266 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.463901 (+ 1635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.463903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.463920 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.463996 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.464001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.465248 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.465252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.466255 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.466264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.467162 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.467166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.467252 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.467300 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.467380 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.467457 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":23,"thread_start_us":101,"threads_started":1}
I20260810 07:15:32.467633 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.462039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.12.117.129:38829)
0810 07:15:32.463549 (+ 1510us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.463564 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.463586 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:15:32.463647 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.463650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.463776 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.463925 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.464087 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.465150 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.466361 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.467037 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.467153 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.467298 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.467409 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.467497 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1411,"thread_start_us":82,"threads_started":1}
I20260810 07:15:32.468137 25305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.462248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:43867 (local address 127.0.0.1:59568)
0810 07:15:32.462468 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.462481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.462553 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.463612 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.463615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.463634 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.463720 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.463727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.465088 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.465091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.466547 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.466559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.467768 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.467771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.467864 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.467882 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.467947 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.468016 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":30,"thread_start_us":95,"threads_started":1}
I20260810 07:15:32.468214 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.462639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59568 (local address 127.12.117.131:43867)
0810 07:15:32.463053 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.463064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.463086 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:15:32.463157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.463162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.463356 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.463490 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.463836 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.464960 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.466691 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.467473 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.467584 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.467701 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.468002 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.468091 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":90,"threads_started":1}
I20260810 07:15:32.469781 24925 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab4c8eb2bd341789654089518e10b57 (DEFAULT_TABLE table=test-table [id=1b43b6d6ebc44437a48460033e45a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:32.470358 24675 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab4c8eb2bd341789654089518e10b57 (DEFAULT_TABLE table=test-table [id=1b43b6d6ebc44437a48460033e45a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:32.470888 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ab4c8eb2bd341789654089518e10b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:32.471138 24675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ab4c8eb2bd341789654089518e10b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:32.471189 25145 tablet_service.cc:1467] Processing CreateTablet for tablet 9ab4c8eb2bd341789654089518e10b57 (DEFAULT_TABLE table=test-table [id=1b43b6d6ebc44437a48460033e45a0f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:32.472131 25145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ab4c8eb2bd341789654089518e10b57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:32.477212 25308 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Bootstrap starting.
I20260810 07:15:32.479357 25308 tablet_bootstrap.cc:654] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:32.479578 25309 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Bootstrap starting.
I20260810 07:15:32.479853 25310 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Bootstrap starting.
I20260810 07:15:32.480418 25308 log.cc:824] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:32.481539 25309 tablet_bootstrap.cc:654] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:32.481707 25310 tablet_bootstrap.cc:654] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:32.482329 25310 log.cc:824] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:32.482628 25309 log.cc:824] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:32.482707 25308 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: No bootstrap required, opened a new log
I20260810 07:15:32.482882 25308 ts_tablet_manager.cc:1397] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 07:15:32.484283 25310 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: No bootstrap required, opened a new log
I20260810 07:15:32.484534 25309 tablet_bootstrap.cc:492] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: No bootstrap required, opened a new log
I20260810 07:15:32.484578 25310 ts_tablet_manager.cc:1397] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:32.484700 25309 ts_tablet_manager.cc:1397] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:15:32.486824 25308 raft_consensus.cc:348] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.487020 25308 raft_consensus.cc:374] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:32.487072 25308 raft_consensus.cc:729] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801f9248a6ea48829543c852775bbd1b, State: Initialized, Role: FOLLOWER
I20260810 07:15:32.487494 25308 consensus_queue.cc:260] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.488387 25308 ts_tablet_manager.cc:1428] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:15:32.488627 24792 heartbeater.cc:503] Master 127.12.117.190:41229 was elected leader, sending a full tablet report...
I20260810 07:15:32.488942 25310 raft_consensus.cc:348] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.489176 25310 raft_consensus.cc:374] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:32.489300 25310 raft_consensus.cc:729] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd6d9d25ade4fcf9bfe88a2b81349d3, State: Initialized, Role: FOLLOWER
I20260810 07:15:32.489442 25309 raft_consensus.cc:348] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.489674 25309 raft_consensus.cc:374] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:32.489727 25309 raft_consensus.cc:729] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490e07200bcc4287bfc4af6e1a9753e3, State: Initialized, Role: FOLLOWER
I20260810 07:15:32.489777 25310 consensus_queue.cc:260] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.490146 25309 consensus_queue.cc:260] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.490864 25310 ts_tablet_manager.cc:1428] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:15:32.491065 25309 ts_tablet_manager.cc:1428] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260810 07:15:32.492017 25027 heartbeater.cc:503] Master 127.12.117.190:41229 was elected leader, sending a full tablet report...
I20260810 07:15:32.492044 25258 heartbeater.cc:503] Master 127.12.117.190:41229 was elected leader, sending a full tablet report...
W20260810 07:15:32.667363 25261 tablet.cc:2369] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:32.702099 25030 tablet.cc:2369] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:32.732802 24794 tablet.cc:2369] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:32.793433 25319 raft_consensus.cc:482] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:32.793642 25319 raft_consensus.cc:504] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.794704 25319 leader_election.cc:288] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829), bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:32.798533 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.795060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:39331 (local address 127.12.117.129:38829)
0810 07:15:32.795202 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.795207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.795359 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:15:32.795513 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.795517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.795568 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.795648 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.795926 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.796503 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.797592 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.797999 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.798092 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.798174 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.798336 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.798409 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:15:32.798533 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.794995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38829 (local address 127.12.117.130:39331)
0810 07:15:32.795241 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.795254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.795400 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.795666 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.795670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.795695 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.795810 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.795816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.796611 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.796614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.797464 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.797474 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.798117 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.798120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.798206 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.798231 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.798308 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.798374 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260810 07:15:32.798682 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.795502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:39489 (local address 127.12.117.131:43867)
0810 07:15:32.795634 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.795639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.795683 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:15:32.795778 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.795782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.795828 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.795907 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:32.796155 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.796675 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.797660 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.798107 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.798217 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.798263 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.798496 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.798575 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:15:32.798699 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.795389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:43867 (local address 127.12.117.130:39489)
0810 07:15:32.795600 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:32.795624 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:32.795685 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:32.795919 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:32.795922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:32.795938 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:32.796043 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.796049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.796788 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.796791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.797537 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:32.797547 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.798244 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.798246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:32.798338 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.798363 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:15:32.798436 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 07:15:32.798512 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260810 07:15:32.799414 25178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab4c8eb2bd341789654089518e10b57" candidate_uuid: "490e07200bcc4287bfc4af6e1a9753e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" is_pre_election: true
I20260810 07:15:32.799413 24705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab4c8eb2bd341789654089518e10b57" candidate_uuid: "490e07200bcc4287bfc4af6e1a9753e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801f9248a6ea48829543c852775bbd1b" is_pre_election: true
I20260810 07:15:32.799724 24705 raft_consensus.cc:2454] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490e07200bcc4287bfc4af6e1a9753e3 in term 0.
I20260810 07:15:32.799724 25178 raft_consensus.cc:2454] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 490e07200bcc4287bfc4af6e1a9753e3 in term 0.
I20260810 07:15:32.800172 24838 leader_election.cc:302] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490e07200bcc4287bfc4af6e1a9753e3, 801f9248a6ea48829543c852775bbd1b; no voters:
I20260810 07:15:32.800416 25319 raft_consensus.cc:2781] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:32.800508 25319 raft_consensus.cc:482] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:32.800549 25319 raft_consensus.cc:3037] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:32.801625 25319 raft_consensus.cc:504] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.801934 25319 leader_election.cc:288] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [CANDIDATE]: Term 1 election: Requested vote from peers 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829), bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867)
I20260810 07:15:32.802268 24705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab4c8eb2bd341789654089518e10b57" candidate_uuid: "490e07200bcc4287bfc4af6e1a9753e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801f9248a6ea48829543c852775bbd1b"
I20260810 07:15:32.802357 25178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ab4c8eb2bd341789654089518e10b57" candidate_uuid: "490e07200bcc4287bfc4af6e1a9753e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3"
I20260810 07:15:32.802399 24705 raft_consensus.cc:3037] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:32.802472 25178 raft_consensus.cc:3037] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:32.803721 24705 raft_consensus.cc:2454] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490e07200bcc4287bfc4af6e1a9753e3 in term 1.
I20260810 07:15:32.804008 24838 leader_election.cc:302] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490e07200bcc4287bfc4af6e1a9753e3, 801f9248a6ea48829543c852775bbd1b; no voters:
I20260810 07:15:32.804158 25178 raft_consensus.cc:2454] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 490e07200bcc4287bfc4af6e1a9753e3 in term 1.
I20260810 07:15:32.804185 25319 raft_consensus.cc:2781] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:32.804365 25319 raft_consensus.cc:686] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 LEADER]: Becoming Leader. State: Replica: 490e07200bcc4287bfc4af6e1a9753e3, State: Running, Role: LEADER
I20260810 07:15:32.804685 25319 consensus_queue.cc:237] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:32.807727 23095 catalog_manager.cc:5495] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130). New cstate: current_term: 1 leader_uuid: "490e07200bcc4287bfc4af6e1a9753e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:32.834892 25301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.831686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54612 (local address 127.12.117.130:45751)
0810 07:15:32.831822 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.831826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.831839 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:32.831936 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.831938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.831987 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.832065 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:32.832223 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.832696 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.833573 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.833791 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.833867 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.834162 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:32.834578 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:32.834679 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.834722 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.834769 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:15:32.840492 25178 raft_consensus.cc:1261] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Refusing update from remote peer 490e07200bcc4287bfc4af6e1a9753e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:32.840492 24705 raft_consensus.cc:1261] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Refusing update from remote peer 490e07200bcc4287bfc4af6e1a9753e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:32.841205 25319 consensus_queue.cc:1035] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:32.841403 25323 consensus_queue.cc:1035] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:32.850385 25333 mvcc.cc:205] Tried to move back new op lower bound from 7316873760107548672 to 7316873759970148352. Current Snapshot: MvccSnapshot[applied={T|T < 7316873760107548672}]
I20260810 07:15:32.861477 23095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:32.861622 23095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:32.865495 23095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8ae3d13348644b983abe5432ea684dc: Sending DeleteTablet for 3 replicas of tablet 9ab4c8eb2bd341789654089518e10b57
I20260810 07:15:32.866274 24925 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ab4c8eb2bd341789654089518e10b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:32 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:54610
I20260810 07:15:32.866443 25145 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ab4c8eb2bd341789654089518e10b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:32 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59568
I20260810 07:15:32.866501 24675 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ab4c8eb2bd341789654089518e10b57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:32 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:46376
I20260810 07:15:32.866832 25347 tablet_replica.cc:330] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: stopping tablet replica
I20260810 07:15:32.867148 25347 raft_consensus.cc:2229] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:32.867478 25347 raft_consensus.cc:2258] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:32.868960 25347 ts_tablet_manager.cc:1905] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:32.869133 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 23034
I20260810 07:15:32.880769 25349 tablet_replica.cc:330] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: stopping tablet replica
I20260810 07:15:32.881093 25349 raft_consensus.cc:2229] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:32.881404 25349 raft_consensus.cc:2258] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:32.881987 25347 ts_tablet_manager.cc:1918] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:32.882076 25347 log.cc:1196] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/9ab4c8eb2bd341789654089518e10b57
I20260810 07:15:32.882398 25347 ts_tablet_manager.cc:1939] T 9ab4c8eb2bd341789654089518e10b57 P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting consensus metadata
I20260810 07:15:32.884322 25345 tablet_replica.cc:330] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: stopping tablet replica
I20260810 07:15:32.884600 25345 raft_consensus.cc:2229] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:32.884831 25349 ts_tablet_manager.cc:1905] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:32.885031 25345 raft_consensus.cc:2258] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:32.886415 25345 ts_tablet_manager.cc:1905] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:32.887550 25349 ts_tablet_manager.cc:1918] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:32.887653 25349 log.cc:1196] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/9ab4c8eb2bd341789654089518e10b57
I20260810 07:15:32.888013 25349 ts_tablet_manager.cc:1939] T 9ab4c8eb2bd341789654089518e10b57 P 801f9248a6ea48829543c852775bbd1b: Deleting consensus metadata
W20260810 07:15:32.889132 25067 connection.cc:535] server connection from 127.0.0.1:59568 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:15:32.889571 24595 connection.cc:535] server connection from 127.0.0.1:46376 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:15:32.889753 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.868467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41190 (local address 127.12.117.189:43029)
0810 07:15:32.868571 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.868575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.868725 (+ 150us) server_negotiation.cc:407] Connection header received
0810 07:15:32.868961 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.868964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.869021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.869070 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.881685 (+ 12615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.882537 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.883690 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.884009 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.884103 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.886058 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.886062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.886065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.886090 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.887711 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.888183 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.888187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.888189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.888271 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.888553 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.888557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.888559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.888816 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.888929 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.889469 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.889593 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:15:32.891328 25345 ts_tablet_manager.cc:1918] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:32.891430 25345 log.cc:1196] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/9ab4c8eb2bd341789654089518e10b57
I20260810 07:15:32.891781 25345 ts_tablet_manager.cc:1939] T 9ab4c8eb2bd341789654089518e10b57 P 490e07200bcc4287bfc4af6e1a9753e3: Deleting consensus metadata
W20260810 07:15:32.893249 24839 connection.cc:441] server connection from 127.0.0.1:54610 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54610 (request call id 1) could send its response
I20260810 07:15:32.903688 24805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.867541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.12.117.188:45423)
0810 07:15:32.867650 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.867654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.890150 (+ 22496us) server_negotiation.cc:407] Connection header received
0810 07:15:32.890369 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.890372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.890424 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.890474 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.891508 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.892301 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.893310 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.893579 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.893639 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.895441 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.895444 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.895446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.895468 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.900382 (+ 4914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.900834 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.900838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.900840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.900931 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.901255 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.901259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.901260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.901482 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.901609 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.903422 (+ 1813us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.903539 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:15:32.904191 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.894258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.12.117.188:45423)
0810 07:15:32.895760 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.895766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.895776 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:15:32.895808 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.895811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.895851 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.896686 (+ 835us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.896776 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.897477 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.898646 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.898989 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.899433 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.900483 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.900487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.900500 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.900524 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.902297 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.902752 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.902755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.902756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.902834 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.903114 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.903117 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.903119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.903331 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.903855 (+ 524us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.903929 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.904040 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1444,"thread_start_us":102,"threads_started":1}
I20260810 07:15:32.906237 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:32.895075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41202 (local address 127.12.117.189:43029)
0810 07:15:32.895195 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:32.895199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:32.895392 (+ 193us) server_negotiation.cc:407] Connection header received
0810 07:15:32.895543 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:32.895546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:32.895584 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:32.895628 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:32.896785 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.897558 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.898812 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:32.899152 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:32.899224 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:32.901225 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:32.901229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:32.901232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:32.901256 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:32.903206 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.905085 (+ 1879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.905089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.905090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.905176 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:32.905430 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:32.905433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:32.905435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:32.905645 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:32.905766 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:32.905982 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 07:15:32.906092 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:15:33.854238 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.853754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:52095)
0810 07:15:33.854053 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.854144 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
W20260810 07:15:33.854653 24792 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:41229 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:33.854707 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.854232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:43265)
0810 07:15:33.854515 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.854609 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":64,"threads_started":1}
W20260810 07:15:33.854945 25258 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:41229 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:33.857987 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:33.857636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:59125)
0810 07:15:33.857836 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:33.857909 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
W20260810 07:15:33.858266 25027 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:41229 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:34.366225 25367 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f8ae3d13348644b983abe5432ea684dc)
I20260810 07:15:34.366371 25369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f8ae3d13348644b983abe5432ea684dc)
I20260810 07:15:34.366454 25367 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:34.366544 25369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:34.367447 25367 leader_election.cc:288] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), 3d88e98d411d41038f21fc4875501737 (127.12.117.189:43029)
I20260810 07:15:34.368036 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46042)
0810 07:15:34.367872 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.367943 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":59,"threads_started":1}
I20260810 07:15:34.368036 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46034)
0810 07:15:34.367824 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.367897 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":58,"threads_started":1}
I20260810 07:15:34.368036 25369 leader_election.cc:288] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), f17cb3ec531249839cb94887948a331c (127.12.117.188:45423)
W20260810 07:15:34.368213 24114 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
W20260810 07:15:34.368227 24452 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:34.369169 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.368898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46046)
0810 07:15:34.369057 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.369106 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 07:15:34.369302 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.369000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46056)
0810 07:15:34.369143 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.369192 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 07:15:34.369359 24114 leader_election.cc:334] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
W20260810 07:15:34.369382 24452 leader_election.cc:334] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:34.371856 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41204 (local address 127.12.117.189:43029)
0810 07:15:34.368741 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:34.368745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:34.368758 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:34.368800 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:34.368803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:34.368845 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:34.369205 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:34.369307 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.369949 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.370954 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.371382 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.371481 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.371534 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:34.371694 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 07:15:34.371749 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":823,"thread_start_us":60,"threads_started":1}
I20260810 07:15:34.371873 25382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.12.117.188:45423)
0810 07:15:34.368253 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:34.368258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:34.368568 (+ 310us) server_negotiation.cc:407] Connection header received
0810 07:15:34.368623 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:34.368626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:34.368670 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:34.368743 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:34.369130 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.369873 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.370909 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.371375 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.371482 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.371540 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:34.371670 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 07:15:34.371732 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":280,"thread_start_us":70,"threads_started":1}
I20260810 07:15:34.371884 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:43029 (local address 127.0.0.1:41204)
0810 07:15:34.368360 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.368374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:34.368483 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:34.368978 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:34.368981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:34.369000 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:34.369101 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.369107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.370041 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.370043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.370834 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.370843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.371487 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.371490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.371572 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.371590 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:15:34.371661 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:15:34.371737 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":1,"thread_start_us":138,"threads_started":1}
I20260810 07:15:34.371868 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.367582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45423 (local address 127.0.0.1:46434)
0810 07:15:34.368371 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.368385 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:34.368489 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:34.368875 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:34.368878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:34.368899 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:34.369007 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.369014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.369972 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.369976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.370795 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:34.370805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:34.371481 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:34.371484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:34.371566 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:34.371586 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:15:34.371647 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 07:15:34.371724 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":59,"threads_started":1}
I20260810 07:15:34.372321 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f17cb3ec531249839cb94887948a331c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d88e98d411d41038f21fc4875501737" is_pre_election: true
I20260810 07:15:34.372510 24153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f17cb3ec531249839cb94887948a331c in term 1.
I20260810 07:15:34.372633 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d88e98d411d41038f21fc4875501737" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f17cb3ec531249839cb94887948a331c" is_pre_election: true
I20260810 07:15:34.372815 24500 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d88e98d411d41038f21fc4875501737 in term 1.
I20260810 07:15:34.372866 24453 leader_election.cc:302] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d88e98d411d41038f21fc4875501737, f17cb3ec531249839cb94887948a331c; no voters: f8ae3d13348644b983abe5432ea684dc
I20260810 07:15:34.373072 25367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:15:34.373132 25367 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Starting leader election (detected failure of leader f8ae3d13348644b983abe5432ea684dc)
I20260810 07:15:34.373170 25367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:34.373148 24113 leader_election.cc:302] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d88e98d411d41038f21fc4875501737, f17cb3ec531249839cb94887948a331c; no voters: f8ae3d13348644b983abe5432ea684dc
I20260810 07:15:34.373329 25369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 07:15:34.373376 25369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f8ae3d13348644b983abe5432ea684dc)
I20260810 07:15:34.373410 25369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:34.374445 25367 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:34.374445 25369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:34.374799 25367 leader_election.cc:288] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 election: Requested vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), 3d88e98d411d41038f21fc4875501737 (127.12.117.189:43029)
I20260810 07:15:34.375138 25369 leader_election.cc:288] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 election: Requested vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), f17cb3ec531249839cb94887948a331c (127.12.117.188:45423)
I20260810 07:15:34.375197 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f17cb3ec531249839cb94887948a331c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d88e98d411d41038f21fc4875501737"
I20260810 07:15:34.375344 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.374949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46066)
0810 07:15:34.375211 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.375260 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202}
I20260810 07:15:34.375366 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.374918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46064)
0810 07:15:34.375063 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.375312 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 07:15:34.375389 24153 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f17cb3ec531249839cb94887948a331c in current term 2: Already voted for candidate 3d88e98d411d41038f21fc4875501737 in this term.
I20260810 07:15:34.375355 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d88e98d411d41038f21fc4875501737" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f17cb3ec531249839cb94887948a331c"
I20260810 07:15:34.375535 24500 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3d88e98d411d41038f21fc4875501737 in current term 2: Already voted for candidate f17cb3ec531249839cb94887948a331c in this term.
I20260810 07:15:34.375976 25373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.375732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46072)
0810 07:15:34.375868 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.375914 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 07:15:34.375976 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.375746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46088)
0810 07:15:34.375868 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.375928 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 07:15:34.376099 24114 leader_election.cc:334] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
W20260810 07:15:34.376150 24452 leader_election.cc:334] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:34.376175 24114 leader_election.cc:302] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3d88e98d411d41038f21fc4875501737; no voters: f17cb3ec531249839cb94887948a331c, f8ae3d13348644b983abe5432ea684dc
I20260810 07:15:34.376214 24452 leader_election.cc:302] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f17cb3ec531249839cb94887948a331c; no voters: 3d88e98d411d41038f21fc4875501737, f8ae3d13348644b983abe5432ea684dc
I20260810 07:15:34.376312 25369 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:15:34.376329 25367 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 07:15:34.855948 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.855473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:57017)
0810 07:15:34.855772 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.855840 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":71,"threads_started":1}
I20260810 07:15:34.856043 25387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.855216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:46915)
0810 07:15:34.855848 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.855939 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":114,"threads_started":1}
I20260810 07:15:34.859205 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:34.858770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:46655)
0810 07:15:34.859040 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:34.859109 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260810 07:15:35.857064 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.856674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:54247)
0810 07:15:35.856918 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.856973 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260810 07:15:35.857095 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.856709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:54997)
0810 07:15:35.856926 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.857009 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
I20260810 07:15:35.860237 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:35.859799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:42091)
0810 07:15:35.860059 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:35.860142 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":101,"threads_started":1}
I20260810 07:15:36.184875 25401 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:36.184999 25401 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:36.185400 25401 leader_election.cc:288] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), 3d88e98d411d41038f21fc4875501737 (127.12.117.189:43029)
I20260810 07:15:36.185799 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f17cb3ec531249839cb94887948a331c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d88e98d411d41038f21fc4875501737" is_pre_election: true
I20260810 07:15:36.185945 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.185551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46382)
0810 07:15:36.185806 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.185858 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":60,"threads_started":1}
I20260810 07:15:36.185953 24153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f17cb3ec531249839cb94887948a331c in term 2.
I20260810 07:15:36.186267 24453 leader_election.cc:302] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d88e98d411d41038f21fc4875501737, f17cb3ec531249839cb94887948a331c; no voters:
I20260810 07:15:36.186435 25401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 07:15:36.186499 25401 raft_consensus.cc:482] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:36.186542 25401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:15:36.186789 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.186579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46386)
0810 07:15:36.186679 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.186730 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:15:36.186900 24452 leader_election.cc:334] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:36.187719 25401 raft_consensus.cc:504] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:36.188025 25401 leader_election.cc:288] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 election: Requested vote from peers f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229), 3d88e98d411d41038f21fc4875501737 (127.12.117.189:43029)
I20260810 07:15:36.188356 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.188128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46388)
0810 07:15:36.188272 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.188310 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 07:15:36.188401 24153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f17cb3ec531249839cb94887948a331c" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "3d88e98d411d41038f21fc4875501737"
I20260810 07:15:36.188535 24153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:15:36.188970 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.188761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46402)
0810 07:15:36.188879 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.188924 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 07:15:36.189077 24452 leader_election.cc:334] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
I20260810 07:15:36.189590 24153 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f17cb3ec531249839cb94887948a331c in term 3.
I20260810 07:15:36.189841 24453 leader_election.cc:302] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3d88e98d411d41038f21fc4875501737, f17cb3ec531249839cb94887948a331c; no voters: f8ae3d13348644b983abe5432ea684dc
I20260810 07:15:36.189996 25401 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:15:36.190198 25401 raft_consensus.cc:686] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [term 3 LEADER]: Becoming Leader. State: Replica: f17cb3ec531249839cb94887948a331c, State: Running, Role: LEADER
I20260810 07:15:36.190454 25401 consensus_queue.cc:237] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } }
I20260810 07:15:36.191474 25406 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17cb3ec531249839cb94887948a331c. Latest consensus state: current_term: 3 leader_uuid: "f17cb3ec531249839cb94887948a331c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:36.191591 25406 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:36.191879 25408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:36.194084 25408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:36.194473 25408 catalog_manager.cc:1259] Loaded cluster ID: 80aca1c9dc494a7ebde5fc8c678672eb
I20260810 07:15:36.194517 25408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:36.195062 25408 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:36.195487 25408 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Loaded TSK: 0
I20260810 07:15:36.196142 25408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:36.255291 24485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:36.258292 24485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:36.261152 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.260887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46414)
0810 07:15:36.261038 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.261087 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 07:15:36.261220 24153 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [term 3 FOLLOWER]: Refusing update from remote peer f17cb3ec531249839cb94887948a331c: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 07:15:36.261694 25406 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 07:15:36.261915 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.261673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.0.0.1:46420)
0810 07:15:36.261764 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.261813 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:15:36.262038 24452 consensus_peers.cc:487] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c -> Peer f8ae3d13348644b983abe5432ea684dc (127.12.117.190:41229): Couldn't send request to peer f8ae3d13348644b983abe5432ea684dc. Status: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 07:15:36.263123 25410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f17cb3ec531249839cb94887948a331c. Latest consensus state: current_term: 3 leader_uuid: "f17cb3ec531249839cb94887948a331c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:36.263237 25410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:36.264228 25406 sys_catalog.cc:455] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f17cb3ec531249839cb94887948a331c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:36.264345 25406 sys_catalog.cc:458] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:36.264834 25410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f17cb3ec531249839cb94887948a331c" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f8ae3d13348644b983abe5432ea684dc" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 41229 } } peers { permanent_uuid: "3d88e98d411d41038f21fc4875501737" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 43029 } attrs { promote: false } } peers { permanent_uuid: "f17cb3ec531249839cb94887948a331c" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45423 } attrs { promote: false } } }
I20260810 07:15:36.264937 25410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d88e98d411d41038f21fc4875501737 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:15:36.274237 25402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.269789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:45751 (local address 127.0.0.1:54626)
0810 07:15:36.269950 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.269964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.270052 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.270375 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.270378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.270410 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:36.270531 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.270538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.271698 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.271701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.272836 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.272845 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.273681 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.273684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.273972 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.273998 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:15:36.274060 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 07:15:36.274128 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":164,"spinlock_wait_cycles":3456}
I20260810 07:15:36.274466 25422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.270418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:43867 (local address 127.0.0.1:59584)
0810 07:15:36.270857 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.270869 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.270957 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.271319 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.271323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.271340 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:36.271428 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.271435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.272240 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.272243 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.273206 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.273216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.274096 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.274100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.274186 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.274210 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:15:36.274285 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 07:15:36.274354 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":72,"threads_started":1}
I20260810 07:15:36.274533 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.270824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.12.117.131:43867)
0810 07:15:36.271082 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.271086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.271100 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:36.271154 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.271156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.271200 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.271312 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:36.271554 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.272131 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.273344 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.273966 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.274078 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.274125 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.274317 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.274412 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
I20260810 07:15:36.274753 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.269877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.12.117.130:45751)
0810 07:15:36.270115 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.270121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.270135 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:36.270191 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.270194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.270246 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.270644 (+ 398us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:36.270758 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.271600 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.272959 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.273546 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.273813 (+ 267us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.273868 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.274100 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.274181 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260810 07:15:36.275012 25145 tablet_service.cc:1467] Processing CreateTablet for tablet d17259bf626b48bd915194da102860ad (DEFAULT_TABLE table=test-table [id=206502086b0a4cc5bb7c152863ac0f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:36.275344 25145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17259bf626b48bd915194da102860ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:36.275477 24925 tablet_service.cc:1467] Processing CreateTablet for tablet d17259bf626b48bd915194da102860ad (DEFAULT_TABLE table=test-table [id=206502086b0a4cc5bb7c152863ac0f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:36.275661 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.270102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38829 (local address 127.0.0.1:46390)
0810 07:15:36.270338 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.270352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.270441 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.271506 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.271509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.271528 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:36.271619 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.271624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.272703 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.272706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.273810 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.273820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.275250 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.275253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.275374 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.275399 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:15:36.275478 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:15:36.275545 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":28,"thread_start_us":86,"threads_started":1}
I20260810 07:15:36.275702 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.270735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46390 (local address 127.12.117.129:38829)
0810 07:15:36.271246 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.271251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.271282 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:15:36.271331 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.271334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.271387 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.271722 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:36.271828 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.272587 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.274472 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.275085 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.275254 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.275324 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.275508 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.275583 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":61,"threads_started":1}
I20260810 07:15:36.275732 24925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17259bf626b48bd915194da102860ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:36.277820 25425 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Bootstrap starting.
I20260810 07:15:36.277830 25429 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Bootstrap starting.
I20260810 07:15:36.278041 24675 tablet_service.cc:1467] Processing CreateTablet for tablet d17259bf626b48bd915194da102860ad (DEFAULT_TABLE table=test-table [id=206502086b0a4cc5bb7c152863ac0f26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:36.278323 24675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d17259bf626b48bd915194da102860ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:36.278714 25425 tablet_bootstrap.cc:654] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:36.278931 25429 tablet_bootstrap.cc:654] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:36.279879 25425 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: No bootstrap required, opened a new log
I20260810 07:15:36.279975 25425 ts_tablet_manager.cc:1397] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:36.280238 25429 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: No bootstrap required, opened a new log
I20260810 07:15:36.280330 25429 ts_tablet_manager.cc:1397] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260810 07:15:36.280339 25425 raft_consensus.cc:348] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.280444 25425 raft_consensus.cc:374] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:36.280478 25425 raft_consensus.cc:729] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 490e07200bcc4287bfc4af6e1a9753e3, State: Initialized, Role: FOLLOWER
I20260810 07:15:36.280598 25425 consensus_queue.cc:260] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.280732 25429 raft_consensus.cc:348] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.280838 25429 raft_consensus.cc:374] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:36.280879 25429 raft_consensus.cc:729] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd6d9d25ade4fcf9bfe88a2b81349d3, State: Initialized, Role: FOLLOWER
I20260810 07:15:36.280956 25425 ts_tablet_manager.cc:1428] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:36.281021 25429 consensus_queue.cc:260] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.281136 25434 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: Bootstrap starting.
I20260810 07:15:36.281358 25429 ts_tablet_manager.cc:1428] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:36.282580 25434 tablet_bootstrap.cc:654] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:36.282887 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.282280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:52681)
0810 07:15:36.282769 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.282826 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416}
I20260810 07:15:36.283254 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.281625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:45617)
0810 07:15:36.283142 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.283195 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1447}
I20260810 07:15:36.283809 25025 heartbeater.cc:503] Master 127.12.117.188:45423 was elected leader, sending a full tablet report...
I20260810 07:15:36.284065 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.283712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:39551)
0810 07:15:36.283960 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.284008 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260810 07:15:36.284590 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.284325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:58965)
0810 07:15:36.284478 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.284531 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 07:15:36.285010 25434 tablet_bootstrap.cc:492] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: No bootstrap required, opened a new log
I20260810 07:15:36.285077 25257 heartbeater.cc:503] Master 127.12.117.188:45423 was elected leader, sending a full tablet report...
I20260810 07:15:36.285104 25434 ts_tablet_manager.cc:1397] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 07:15:36.285530 25434 raft_consensus.cc:348] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.285645 25434 raft_consensus.cc:374] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:36.285681 25434 raft_consensus.cc:729] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801f9248a6ea48829543c852775bbd1b, State: Initialized, Role: FOLLOWER
I20260810 07:15:36.285800 25434 consensus_queue.cc:260] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.286108 25437 raft_consensus.cc:482] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:36.286173 25434 ts_tablet_manager.cc:1428] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:36.286271 25437 raft_consensus.cc:504] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.287184 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.286960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:51775)
0810 07:15:36.287061 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.287118 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 07:15:36.287411 25437 leader_election.cc:288] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751), 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:36.288038 24791 heartbeater.cc:503] Master 127.12.117.188:45423 was elected leader, sending a full tablet report...
I20260810 07:15:36.289145 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.288881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:43261)
0810 07:15:36.289037 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.289084 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 07:15:36.292309 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.288117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:45833 (local address 127.12.117.130:45751)
0810 07:15:36.288214 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.288218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.288230 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:36.288269 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.288272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.288312 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.288395 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:36.288749 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.289451 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.290897 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.291560 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.291666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.291712 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.292134 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.292208 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:15:36.292309 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.287891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:48201 (local address 127.12.117.129:38829)
0810 07:15:36.288000 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.288003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.288133 (+ 130us) server_negotiation.cc:407] Connection header received
0810 07:15:36.288261 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.288264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.288304 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.288363 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:36.288612 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.289350 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.290948 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.291560 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.291666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.291712 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.292135 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.292208 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:36.292315 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.287565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:45751 (local address 127.12.117.131:45833)
0810 07:15:36.287667 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.287719 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.287803 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.288506 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.288509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.288528 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:36.288638 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.288644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.289734 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.289738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.290765 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.290777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.291781 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.291784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.291975 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.292020 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:15:36.292085 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:15:36.292183 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":111}
I20260810 07:15:36.292318 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.287793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38829 (local address 127.12.117.131:48201)
0810 07:15:36.288040 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.288052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:36.288143 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:36.288380 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:36.288383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:36.288407 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:36.288512 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.288519 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.289510 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.289513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.290788 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:36.290799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.291776 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.291780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:36.291977 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.292013 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 07:15:36.292090 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:15:36.292185 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":76,"thread_start_us":95,"threads_started":1}
I20260810 07:15:36.292789 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17259bf626b48bd915194da102860ad" candidate_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490e07200bcc4287bfc4af6e1a9753e3" is_pre_election: true
I20260810 07:15:36.292999 24954 raft_consensus.cc:2454] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd6d9d25ade4fcf9bfe88a2b81349d3 in term 0.
I20260810 07:15:36.293155 24705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17259bf626b48bd915194da102860ad" candidate_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801f9248a6ea48829543c852775bbd1b" is_pre_election: true
I20260810 07:15:36.293284 24705 raft_consensus.cc:2454] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd6d9d25ade4fcf9bfe88a2b81349d3 in term 0.
I20260810 07:15:36.293382 25062 leader_election.cc:302] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 490e07200bcc4287bfc4af6e1a9753e3, bfd6d9d25ade4fcf9bfe88a2b81349d3; no voters:
I20260810 07:15:36.293608 25437 raft_consensus.cc:2781] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:36.293663 25437 raft_consensus.cc:482] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:36.293690 25437 raft_consensus.cc:3037] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:36.294528 25437 raft_consensus.cc:504] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.294813 25437 leader_election.cc:288] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [CANDIDATE]: Term 1 election: Requested vote from peers 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751), 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829)
I20260810 07:15:36.295207 24705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17259bf626b48bd915194da102860ad" candidate_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801f9248a6ea48829543c852775bbd1b"
I20260810 07:15:36.295239 24954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d17259bf626b48bd915194da102860ad" candidate_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "490e07200bcc4287bfc4af6e1a9753e3"
I20260810 07:15:36.295331 24705 raft_consensus.cc:3037] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:36.295357 24954 raft_consensus.cc:3037] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:36.296118 24705 raft_consensus.cc:2454] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd6d9d25ade4fcf9bfe88a2b81349d3 in term 1.
I20260810 07:15:36.296175 24954 raft_consensus.cc:2454] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd6d9d25ade4fcf9bfe88a2b81349d3 in term 1.
I20260810 07:15:36.296358 25062 leader_election.cc:302] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801f9248a6ea48829543c852775bbd1b, bfd6d9d25ade4fcf9bfe88a2b81349d3; no voters:
I20260810 07:15:36.296495 25437 raft_consensus.cc:2781] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:36.296684 25437 raft_consensus.cc:686] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 LEADER]: Becoming Leader. State: Replica: bfd6d9d25ade4fcf9bfe88a2b81349d3, State: Running, Role: LEADER
I20260810 07:15:36.297097 25437 consensus_queue.cc:237] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } }
I20260810 07:15:36.297211 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.297015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:37063)
0810 07:15:36.297116 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.297162 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:15:36.299059 24483 catalog_manager.cc:5495] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfd6d9d25ade4fcf9bfe88a2b81349d3" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43867 } health_report { overall_health: HEALTHY } } }
I20260810 07:15:36.313963 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.311044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.12.117.131:43867)
0810 07:15:36.311139 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:36.311144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:36.311156 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:36.311294 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:36.311297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:36.311350 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:36.311434 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:36.311612 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.312079 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.312840 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:36.313066 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:36.313139 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:36.313312 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:36.313697 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:36.313775 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:36.313818 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:15:36.313859 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:36.316320 24705 raft_consensus.cc:1261] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Refusing update from remote peer bfd6d9d25ade4fcf9bfe88a2b81349d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:36.316499 24954 raft_consensus.cc:1261] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Refusing update from remote peer bfd6d9d25ade4fcf9bfe88a2b81349d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:36.316802 25443 consensus_queue.cc:1035] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "801f9248a6ea48829543c852775bbd1b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:36.317000 25437 consensus_queue.cc:1035] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "490e07200bcc4287bfc4af6e1a9753e3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:36.318428 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.318200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:42627)
0810 07:15:36.318302 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.318355 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:15:36.320144 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.319243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:47533)
0810 07:15:36.320051 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.320094 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":748}
I20260810 07:15:36.321928 25441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.321711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.131:57465)
0810 07:15:36.321799 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.321853 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:15:36.322585 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.322379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.130:40911)
0810 07:15:36.322481 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.322528 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:15:36.324318 25455 mvcc.cc:205] Tried to move back new op lower bound from 7316873774347067392 to 7316873774274973696. Current Snapshot: MvccSnapshot[applied={T|T < 7316873774347067392}]
I20260810 07:15:36.324674 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.324375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:41229 (local address 127.12.117.129:43559)
0810 07:15:36.324581 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.324625 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:41229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260810 07:15:36.330348 24483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:36.330466 24483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:36.333789 24483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f17cb3ec531249839cb94887948a331c: Sending DeleteTablet for 3 replicas of tablet d17259bf626b48bd915194da102860ad
I20260810 07:15:36.334420 24925 tablet_service.cc:1514] Processing DeleteTablet for tablet d17259bf626b48bd915194da102860ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:36 UTC) from {username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:54626
I20260810 07:15:36.334519 25145 tablet_service.cc:1514] Processing DeleteTablet for tablet d17259bf626b48bd915194da102860ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:36 UTC) from {username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:59584
I20260810 07:15:36.334739 25465 tablet_replica.cc:330] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: stopping tablet replica
I20260810 07:15:36.334733 24675 tablet_service.cc:1514] Processing DeleteTablet for tablet d17259bf626b48bd915194da102860ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:36 UTC) from {username='oryx', principal='oryx/127.12.117.188@KRBTEST.COM'} at 127.0.0.1:46390
I20260810 07:15:36.334858 25465 raft_consensus.cc:2229] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:36.335168 25470 tablet_replica.cc:330] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b: stopping tablet replica
I20260810 07:15:36.335168 25465 raft_consensus.cc:2258] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:36.335408 25467 tablet_replica.cc:330] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: stopping tablet replica
I20260810 07:15:36.335395 25470 raft_consensus.cc:2229] T d17259bf626b48bd915194da102860ad P 801f9248a6ea48829543c852775bbd1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:36.335426 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 24401
I20260810 07:15:36.353313 25467 raft_consensus.cc:2229] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:36.353807 25467 raft_consensus.cc:2258] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:36.356307 25465 ts_tablet_manager.cc:1905] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:36.356470 25467 ts_tablet_manager.cc:1905] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:36.358974 25465 ts_tablet_manager.cc:1918] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:36.359045 25465 log.cc:1196] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/d17259bf626b48bd915194da102860ad
I20260810 07:15:36.359289 25465 ts_tablet_manager.cc:1939] T d17259bf626b48bd915194da102860ad P 490e07200bcc4287bfc4af6e1a9753e3: Deleting consensus metadata
I20260810 07:15:36.359776 25467 ts_tablet_manager.cc:1918] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:36.359947 25467 log.cc:1196] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/d17259bf626b48bd915194da102860ad
I20260810 07:15:36.360333 25467 ts_tablet_manager.cc:1939] T d17259bf626b48bd915194da102860ad P bfd6d9d25ade4fcf9bfe88a2b81349d3: Deleting consensus metadata
I20260810 07:15:36.360426 24451 catalog_manager.cc:4841] TS 490e07200bcc4287bfc4af6e1a9753e3 (127.12.117.130:45751): tablet d17259bf626b48bd915194da102860ad (table test-table [id=206502086b0a4cc5bb7c152863ac0f26]) successfully deleted
I20260810 07:15:36.361455 24451 catalog_manager.cc:4841] TS bfd6d9d25ade4fcf9bfe88a2b81349d3 (127.12.117.131:43867): tablet d17259bf626b48bd915194da102860ad (table test-table [id=206502086b0a4cc5bb7c152863ac0f26]) successfully deleted
I20260810 07:15:36.371160 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 24806
I20260810 07:15:36.371310 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:36.371014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38829 (local address 127.0.0.1:46398)
0810 07:15:36.371164 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:36.371209 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260810 07:15:36.372852 24451 catalog_manager.cc:4584] TS 801f9248a6ea48829543c852775bbd1b (127.12.117.129:38829): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d17259bf626b48bd915194da102860ad: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38829: connect: Connection refused (error 111)
I20260810 07:15:36.381311 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 25038
I20260810 07:15:36.389660 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 23889
I20260810 07:15:36.402076 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 24070
2026-08-10T07:15:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7327 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:36 dist-test-slave-gjrz krb5kdc[25473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:36 dist-test-slave-gjrz krb5kdc[25473](info): set up 2 sockets
Aug 10 07:15:36 dist-test-slave-gjrz krb5kdc[25473](info): commencing operation
krb5kdc: starting...
W20260810 07:15:37.572667 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:37 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab.
Entry for principal oryx/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab.
I20260810 07:15:37.687915 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44969
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab
--principal=oryx/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:37.795778 25500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:37.796006 25500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:37.796059 25500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:37.799961 25500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:37.800004 25500 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 07:15:37.800020 25500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:37.800050 25500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:37.800077 25500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:37.800101 25500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:37.805047 25500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44969
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.190.keytab
--principal=oryx/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:37.805902 25500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:37.806774 25500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:37.814811 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.814810 25508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:37.814855 25510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:37.814998 25500 server_base.cc:1034] running on GCE node
I20260810 07:15:37.815629 25500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:37.816617 25500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:37.817771 25500 hybrid_clock.cc:648] HybridClock initialized: now 1786346137817758 us; error 29 us; skew 500 ppm
Aug 10 07:15:37 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346137, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:37.820267 25500 init.cc:377] Logged in from keytab as oryx/127.12.117.190@KRBTEST.COM (short username oryx)
I20260810 07:15:37.821072 25500 webserver.cc:460] Webserver started at http://127.12.117.190:37391/ using document root <none> and password file <none>
I20260810 07:15:37.821573 25500 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:37.821637 25500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:37.821815 25500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:37.823726 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "8b19119f74604b7b8d010a5481a2133a"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-gjrz"
server_key: "bb751c3e1f547a3b5b3df40948f676d2"
server_key_iv: "a5f843b51f3b1fabab8b410b8fb5780a"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.824245 25500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "8b19119f74604b7b8d010a5481a2133a"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-gjrz"
server_key: "bb751c3e1f547a3b5b3df40948f676d2"
server_key_iv: "a5f843b51f3b1fabab8b410b8fb5780a"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.827657 25500 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:37.829944 25523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:37.831094 25500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:37.831207 25500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "8b19119f74604b7b8d010a5481a2133a"
format_stamp: "Formatted at 2026-08-10 07:15:37 on dist-test-slave-gjrz"
server_key: "bb751c3e1f547a3b5b3df40948f676d2"
server_key_iv: "a5f843b51f3b1fabab8b410b8fb5780a"
server_key_version: "encryptionkey@0"
I20260810 07:15:37.831315 25500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:37.845908 25500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:37.848488 25500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:37.848649 25500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:37.856029 25500 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:44969
I20260810 07:15:37.856038 25628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:44969 every 8 connection(s)
I20260810 07:15:37.856871 25500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:37.859515 25630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:37.862970 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 25500
I20260810 07:15:37.863081 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:37.863373 12758 external_mini_cluster.cc:1428] Setting key 915f3614357e50117117de2362dc5cf8
I20260810 07:15:37.864945 25630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Bootstrap starting.
I20260810 07:15:37.867249 25630 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:37.867970 25630 log.cc:824] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:37.869935 25630 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: No bootstrap required, opened a new log
Aug 10 07:15:37 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:37.873732 25630 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } }
I20260810 07:15:37.873947 25630 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:37.874013 25630 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b19119f74604b7b8d010a5481a2133a, State: Initialized, Role: FOLLOWER
I20260810 07:15:37.874454 25630 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } }
I20260810 07:15:37.874621 25630 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:37.874694 25630 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:37.874789 25630 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:37.875630 25630 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } }
I20260810 07:15:37.875936 25630 leader_election.cc:302] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b19119f74604b7b8d010a5481a2133a; no voters:
I20260810 07:15:37.876232 25630 leader_election.cc:288] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:37.876315 25642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:37.876550 25642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [term 1 LEADER]: Becoming Leader. State: Replica: 8b19119f74604b7b8d010a5481a2133a, State: Running, Role: LEADER
I20260810 07:15:37.876859 25642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } }
I20260810 07:15:37.877256 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:37.864675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35044 (local address 127.12.117.190:44969)
0810 07:15:37.865040 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:37.865056 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:37.865101 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:15:37.865731 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:37.865747 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:37.866051 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:37.866314 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:37.867216 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.868288 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.869417 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:37.869866 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:37.869995 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:37.872571 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:37.872576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:37.872589 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:37.872616 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:37.874170 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:37.874645 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:37.874651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:37.874657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:37.874798 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:37.875082 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:37.875086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:37.875088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:37.875512 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:37.875696 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:37.875967 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 07:15:37.876250 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":101,"threads_started":1}
I20260810 07:15:37.877413 25630 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:37.878037 25642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b19119f74604b7b8d010a5481a2133a. Latest consensus state: current_term: 1 leader_uuid: "8b19119f74604b7b8d010a5481a2133a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } } }
I20260810 07:15:37.878145 25642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:37.878445 25644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b19119f74604b7b8d010a5481a2133a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b19119f74604b7b8d010a5481a2133a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44969 } } }
I20260810 07:15:37.878540 25644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a [sys.catalog]: This master's current role is: LEADER
W20260810 07:15:37.881862 25669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:15:37.881945 25669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:15:37.882074 25658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:37.882980 25658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:37.888262 25658 catalog_manager.cc:1347] Generated new cluster ID: 329b44561b5e4f10af116c9cbb557c0b
I20260810 07:15:37.888327 25658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:37.906061 25658 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:37.906908 25658 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:37.913744 25658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Generated new TSK 0
I20260810 07:15:37.914486 25658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
I20260810 07:15:37.942966 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44969
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:38.051915 25680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:38.052141 25680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:38.052201 25680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:38.056104 25680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:38.056157 25680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:38.056210 25680 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:38.056262 25680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:38.061259 25680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:38.062162 25680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:38.063066 25680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:38.072161 25692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:38.072166 25687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:38.072166 25688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:38.072343 25680 server_base.cc:1034] running on GCE node
I20260810 07:15:38.072784 25680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:38.073376 25680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:38.074538 25680 hybrid_clock.cc:648] HybridClock initialized: now 1786346138074514 us; error 38 us; skew 500 ppm
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:38.077148 25680 init.cc:377] Logged in from keytab as oryx/127.12.117.129@KRBTEST.COM (short username oryx)
I20260810 07:15:38.077951 25680 webserver.cc:460] Webserver started at http://127.12.117.129:41617/ using document root <none> and password file <none>
I20260810 07:15:38.078509 25680 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:38.078572 25680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:38.078748 25680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:38.080673 25680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c7c5dfc63e901638a8a9cfe07d4d4d9f"
server_key_iv: "d122567c70a85b15540938578c554796"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.081152 25680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c7c5dfc63e901638a8a9cfe07d4d4d9f"
server_key_iv: "d122567c70a85b15540938578c554796"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.084554 25680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:38.086716 25700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.087841 25680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:38.087942 25680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c7c5dfc63e901638a8a9cfe07d4d4d9f"
server_key_iv: "d122567c70a85b15540938578c554796"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.088145 25680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:38.106840 25680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:38.109390 25680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:38.109529 25680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:38.110822 25680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:38.110872 25680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.110913 25680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:38.110940 25680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:38.121577 25680 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:42253
I20260810 07:15:38.122406 25680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:38.123157 25912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:42253 every 8 connection(s)
I20260810 07:15:38.124042 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.111630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:44401 (local address 127.12.117.190:44969)
0810 07:15:38.111747 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.111751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.112763 (+ 1012us) server_negotiation.cc:407] Connection header received
0810 07:15:38.113916 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.113920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.113978 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.114036 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.115622 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.116374 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.117527 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.117833 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.117931 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.120561 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.120565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.120568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.120596 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.122043 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.122535 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.122539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.122541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.122631 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.123018 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.123021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.123023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.123253 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.123389 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.123814 (+ 425us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.123920 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:38.124707 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.112116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.129:44401)
0810 07:15:38.112616 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.112671 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.113666 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.114442 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.114451 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.114838 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.115464 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.115485 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.116504 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.116508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.117400 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.117411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.117994 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.117998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.118200 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.118824 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.118844 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.120395 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.122172 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.122177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.122192 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.122434 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.122718 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.122722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.122724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.122897 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.123391 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.123396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.123614 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.123771 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.124000 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":120,"threads_started":1}
I20260810 07:15:38.125846 25913 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44969
I20260810 07:15:38.125986 25913 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:38.126530 25913 heartbeater.cc:511] Master 127.12.117.190:44969 requested a full tablet report, sending...
I20260810 07:15:38.128145 25540 ts_manager.cc:194] Registered new tserver with Master: 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253)
I20260810 07:15:38.129315 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 25680
I20260810 07:15:38.129432 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:38.129623 25540 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:44401
I20260810 07:15:38.129706 12758 external_mini_cluster.cc:1428] Setting key edeff5ec14ba3c128283e5ca576767b5
I20260810 07:15:38.135007 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.128023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:41499 (local address 127.12.117.190:44969)
0810 07:15:38.128136 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.128140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.128282 (+ 142us) server_negotiation.cc:407] Connection header received
0810 07:15:38.128396 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.128399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.128440 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.128514 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.128877 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.129640 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.130709 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.131049 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.131149 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.132277 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.132281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.132283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.132305 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.133546 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.133941 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.133945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.133946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.134023 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.134269 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.134272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.134274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.134442 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.134533 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.134771 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.134874 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:38.135028 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.127934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.129:41499)
0810 07:15:38.128201 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.128213 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.128308 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.128523 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.128525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.128685 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.128776 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.128780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.129824 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.129828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.130579 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.130589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.131259 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.131286 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.131378 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.131811 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.131824 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.132143 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.133651 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.133654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.133656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.133836 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.134106 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.134109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.134111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.134190 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.134533 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.134536 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.134624 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.134710 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.134836 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for oryx/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
I20260810 07:15:38.156283 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44969
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:38.265131 25929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:38.265347 25929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:38.265388 25929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:38.269234 25929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:38.269284 25929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:38.269317 25929 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:38.269352 25929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:38.274215 25929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:38.275027 25929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:38.275902 25929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:38.285526 25941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:38.285549 25935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:38.285607 25936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:38.285800 25929 server_base.cc:1034] running on GCE node
I20260810 07:15:38.286342 25929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:38.286900 25929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:38.288058 25929 hybrid_clock.cc:648] HybridClock initialized: now 1786346138288036 us; error 38 us; skew 500 ppm
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:38.294377 25929 init.cc:377] Logged in from keytab as oryx/127.12.117.130@KRBTEST.COM (short username oryx)
I20260810 07:15:38.295197 25929 webserver.cc:460] Webserver started at http://127.12.117.130:37647/ using document root <none> and password file <none>
I20260810 07:15:38.295718 25929 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:38.295782 25929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:38.295970 25929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:38.297854 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "5aa6f289797f4ef096590b72c41d7b24"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "9e12f9efdc9759c50528e8ccd87cbb8b"
server_key_iv: "115f56cf37c126f2b68bcb07d5d98c4e"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.298378 25929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "5aa6f289797f4ef096590b72c41d7b24"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "9e12f9efdc9759c50528e8ccd87cbb8b"
server_key_iv: "115f56cf37c126f2b68bcb07d5d98c4e"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.301992 25929 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:15:38.304216 25951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.305369 25929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:38.305473 25929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "5aa6f289797f4ef096590b72c41d7b24"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "9e12f9efdc9759c50528e8ccd87cbb8b"
server_key_iv: "115f56cf37c126f2b68bcb07d5d98c4e"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.305563 25929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:38.329598 25929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:38.332556 25929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:38.332724 25929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:38.334081 25929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:38.334141 25929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.334195 25929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:38.334223 25929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:38.344422 25929 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44875
I20260810 07:15:38.344556 26161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44875 every 8 connection(s)
I20260810 07:15:38.345371 25929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:38.347177 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.334805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:55783 (local address 127.12.117.190:44969)
0810 07:15:38.334975 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.334980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.335887 (+ 907us) server_negotiation.cc:407] Connection header received
0810 07:15:38.337094 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.337099 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.337160 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.337248 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.338766 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.339516 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.340643 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.340975 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.341074 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.343603 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.343612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.343614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.343639 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.345092 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.345660 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.345663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.345664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.345764 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.346164 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.346168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.346169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.346392 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.346482 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.346885 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.347031 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:15:38.347911 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.335200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.130:55783)
0810 07:15:38.335721 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.335778 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.336805 (+ 1027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.337465 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.337475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.337935 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.338600 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.338622 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.339641 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.339645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.340509 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.340519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.341138 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.341143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.341292 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.341821 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.341836 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.343420 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.345215 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.345220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.345233 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.345567 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.345881 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.345884 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.345887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.346045 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.346484 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.346489 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.346701 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.346848 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.347094 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":98,"threads_started":1}
I20260810 07:15:38.349071 26163 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44969
I20260810 07:15:38.349211 26163 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:38.349761 26163 heartbeater.cc:511] Master 127.12.117.190:44969 requested a full tablet report, sending...
I20260810 07:15:38.350762 25541 ts_manager.cc:194] Registered new tserver with Master: 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875)
I20260810 07:15:38.351526 25541 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:55783
I20260810 07:15:38.352262 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 25929
I20260810 07:15:38.352391 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:38.352715 12758 external_mini_cluster.cc:1428] Setting key b438d3c5f6bd73ef2f02c2e6f25691a1
I20260810 07:15:38.357728 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.351176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47835 (local address 127.12.117.190:44969)
0810 07:15:38.351307 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.351311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.351443 (+ 132us) server_negotiation.cc:407] Connection header received
0810 07:15:38.351560 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.351563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.351601 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.351681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.352135 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.352698 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.353644 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.353874 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.353964 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.354904 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.354908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.354910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.354936 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.356271 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.356699 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.356701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.356702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.356766 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.357012 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.357014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.357015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.357202 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.357276 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.357537 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.357625 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:38.357775 26177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.351069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.130:47835)
0810 07:15:38.351363 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.351376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.351479 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.351699 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.351701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.351896 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.352035 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.352043 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.352856 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.352859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.353543 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.353554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.353983 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.353987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.354081 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.354482 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.354496 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.354766 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.356379 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.356382 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.356384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.356616 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.356836 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.356838 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.356840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.356937 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.357291 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.357294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.357389 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.357492 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.357613 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for oryx/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
I20260810 07:15:38.379048 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:44969
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:38.493415 26184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:38.493640 26184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:38.493680 26184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:38.497556 26184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:38.497605 26184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:38.497639 26184 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 07:15:38.497674 26184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:38.502557 26184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.117.190:44969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:38.503450 26184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:38.504324 26184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:38.513334 26191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:38.513391 26194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:38.513453 26184 server_base.cc:1034] running on GCE node
W20260810 07:15:38.513334 26190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:38.513935 26184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:38.514510 26184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:38.515676 26184 hybrid_clock.cc:648] HybridClock initialized: now 1786346138515658 us; error 30 us; skew 500 ppm
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:38.518222 26184 init.cc:377] Logged in from keytab as oryx/127.12.117.131@KRBTEST.COM (short username oryx)
I20260810 07:15:38.518976 26184 webserver.cc:460] Webserver started at http://127.12.117.131:41741/ using document root <none> and password file <none>
I20260810 07:15:38.519515 26184 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:38.519577 26184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:38.519743 26184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:38.521832 26184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "96b9d45c053f4d4eaca06e24c17624a4"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c45c4cd4019b258faca195df73e1b963"
server_key_iv: "4cc743cb452a761b1a8f4d25968fa13f"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.522351 26184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "96b9d45c053f4d4eaca06e24c17624a4"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c45c4cd4019b258faca195df73e1b963"
server_key_iv: "4cc743cb452a761b1a8f4d25968fa13f"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.525930 26184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:15:38.528275 26209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.529392 26184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:38.529489 26184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "96b9d45c053f4d4eaca06e24c17624a4"
format_stamp: "Formatted at 2026-08-10 07:15:38 on dist-test-slave-gjrz"
server_key: "c45c4cd4019b258faca195df73e1b963"
server_key_iv: "4cc743cb452a761b1a8f4d25968fa13f"
server_key_version: "encryptionkey@0"
I20260810 07:15:38.529587 26184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:38.553656 26184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:38.556077 26184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:38.556249 26184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:38.557534 26184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:38.557580 26184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:38.557618 26184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:38.557646 26184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:38.567595 26184 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:36327
I20260810 07:15:38.567700 26409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:36327 every 8 connection(s)
I20260810 07:15:38.568499 26184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:38.570561 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.558326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:34893 (local address 127.12.117.190:44969)
0810 07:15:38.558450 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.558453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.559439 (+ 986us) server_negotiation.cc:407] Connection header received
0810 07:15:38.560352 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.560355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.560412 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.560468 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.562155 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.562972 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.564270 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.564502 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.564596 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.567203 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.567207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.567211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.567241 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.568711 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.569294 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.569298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.569300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.569399 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.569700 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.569702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.569703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.569875 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.569949 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.570346 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.570444 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:38.571249 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.558808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.131:34893)
0810 07:15:38.559253 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.559335 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.560117 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.560876 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.560886 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.561324 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.561959 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.561981 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.563110 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.563114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.564151 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.564162 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.564640 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.564645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.564809 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.565342 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.565358 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.567024 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.568818 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.568823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.568838 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.569169 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.569490 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.569493 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.569494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.569623 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.569955 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.569960 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.570164 (+ 204us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.570308 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.570546 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260810 07:15:38.572314 26410 heartbeater.cc:347] Connected to a master server at 127.12.117.190:44969
I20260810 07:15:38.572468 26410 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:38.572980 26410 heartbeater.cc:511] Master 127.12.117.190:44969 requested a full tablet report, sending...
I20260810 07:15:38.574136 25540 ts_manager.cc:194] Registered new tserver with Master: 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:38.574822 25540 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:34893
I20260810 07:15:38.576399 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 26184
I20260810 07:15:38.576526 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:38.576836 12758 external_mini_cluster.cc:1428] Setting key ee7666fe2bb10fa5868bbff559cb9349
I20260810 07:15:38.578981 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:15:38.581228 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.574635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:40869 (local address 127.12.117.190:44969)
0810 07:15:38.574753 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.574757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.574903 (+ 146us) server_negotiation.cc:407] Connection header received
0810 07:15:38.575022 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.575024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.575062 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.575136 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.575628 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.576138 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.576993 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.577266 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.577361 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.578460 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.578463 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.578465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.578487 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.579805 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.580228 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.580230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.580231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.580302 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.580548 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.580550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.580551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.580710 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.580778 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.581013 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.581095 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:15:38.581267 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.574564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.12.117.131:40869)
0810 07:15:38.574821 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.574836 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.574937 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.575160 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.575163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.575380 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:38.575497 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.575504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.576267 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.576270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.576884 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.576895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.577389 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.577392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.577492 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.577941 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:38.577953 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:38.578348 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:38.579911 (+ 1563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.579915 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.579917 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.580143 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.580380 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:38.580382 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:38.580384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:38.580472 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:38.580788 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:38.580790 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:38.580878 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.580981 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.581105 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:38.602249 25638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:38.596873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.12.117.190:44969)
0810 07:15:38.596982 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.596985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.596997 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:38.597026 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.597028 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.597059 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.597123 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.597723 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.598209 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.598947 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.599140 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.599207 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.602123 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35052: BlockingRecv error: recv got EOF from 127.0.0.1:35052 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:15:38 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346138, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:38.610591 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.603462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.12.117.190:44969)
0810 07:15:38.603578 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.603582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.603716 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:15:38.603818 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.603821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.603858 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.603918 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:38.604489 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.604956 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.605708 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.605923 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.606010 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.608168 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:38.608172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:38.608174 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:38.608199 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:38.609251 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.609635 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.609638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.609639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.609709 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:38.609912 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:38.609914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:38.609915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:38.610077 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:38.610174 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.610332 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.610422 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:38.612972 25540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:38.615850 25540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:38.629592 26444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.623696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.0.0.1:60362)
0810 07:15:38.624025 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.624074 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.624204 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.625244 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.625250 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.625274 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:38.625392 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.625399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.626793 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.626797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.628138 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.628149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.629091 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.629199 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629241 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.629345 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.629463 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":24,"thread_start_us":76,"threads_started":1}
I20260810 07:15:38.629658 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.624359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.12.117.129:42253)
0810 07:15:38.624759 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.624775 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.624799 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:15:38.624876 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.624882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.625093 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.625285 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:38.625530 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.626669 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.628257 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.628954 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629082 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629217 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.629389 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.629507 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":81,"threads_started":1}
I20260810 07:15:38.629995 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.623861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44875 (local address 127.0.0.1:42144)
0810 07:15:38.624093 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.624107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.624204 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.626026 (+ 1822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.626029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.626046 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:38.626146 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.626153 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.627370 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.627374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.628496 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.628505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.629542 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.629643 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629668 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.629812 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.629876 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":82,"threads_started":1}
I20260810 07:15:38.630125 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.624396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:36327 (local address 127.0.0.1:41834)
0810 07:15:38.624615 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.624628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.624711 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.625867 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.625870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.625887 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:38.625966 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.625972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.627174 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.627177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.628446 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.628457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.629684 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.629775 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629800 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.629876 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.629946 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":20,"thread_start_us":88,"threads_started":1}
I20260810 07:15:38.630139 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.624776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.12.117.131:36327)
0810 07:15:38.625236 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.625248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.625272 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:15:38.625340 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.625346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.625547 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.625738 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:38.626085 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.627065 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.628580 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.629368 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629498 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629612 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.629923 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.630016 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":97,"threads_started":1}
I20260810 07:15:38.630478 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.625131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42144 (local address 127.12.117.130:44875)
0810 07:15:38.625621 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.625632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.625670 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:15:38.625740 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.625744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.625877 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.626004 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:38.626249 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.627245 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.628616 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.629212 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.629343 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.629479 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.630242 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.630335 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":103,"threads_started":1}
I20260810 07:15:38.632843 26041 tablet_service.cc:1467] Processing CreateTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 (DEFAULT_TABLE table=test-table [id=8d160f396fe34058866ca984b29cba40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.632966 25789 tablet_service.cc:1467] Processing CreateTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 (DEFAULT_TABLE table=test-table [id=8d160f396fe34058866ca984b29cba40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.633073 26295 tablet_service.cc:1467] Processing CreateTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 (DEFAULT_TABLE table=test-table [id=8d160f396fe34058866ca984b29cba40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.633826 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66ac68f5d3b43ecbefdd12ce2821717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.633944 25789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66ac68f5d3b43ecbefdd12ce2821717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.633963 26295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f66ac68f5d3b43ecbefdd12ce2821717. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.640475 26457 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Bootstrap starting.
I20260810 07:15:38.641266 26458 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Bootstrap starting.
I20260810 07:15:38.641999 26460 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Bootstrap starting.
I20260810 07:15:38.642724 26457 tablet_bootstrap.cc:654] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.643388 26458 tablet_bootstrap.cc:654] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.643484 26457 log.cc:824] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:38.644003 26460 tablet_bootstrap.cc:654] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.644210 26458 log.cc:824] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:38.644766 26460 log.cc:824] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:38.645596 26457 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: No bootstrap required, opened a new log
I20260810 07:15:38.645807 26457 ts_tablet_manager.cc:1397] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260810 07:15:38.646184 26458 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: No bootstrap required, opened a new log
I20260810 07:15:38.646371 26458 ts_tablet_manager.cc:1397] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260810 07:15:38.646520 26460 tablet_bootstrap.cc:492] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: No bootstrap required, opened a new log
I20260810 07:15:38.646688 26460 ts_tablet_manager.cc:1397] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:15:38.649796 26457 raft_consensus.cc:348] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.650072 26457 raft_consensus.cc:374] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.650135 26457 raft_consensus.cc:729] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.650028 26460 raft_consensus.cc:348] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.650285 26460 raft_consensus.cc:374] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.650339 26460 raft_consensus.cc:729] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.650382 26458 raft_consensus.cc:348] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.650609 26458 raft_consensus.cc:374] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.650573 26457 consensus_queue.cc:260] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.650669 26458 raft_consensus.cc:729] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.650744 26460 consensus_queue.cc:260] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.651062 26458 consensus_queue.cc:260] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.651470 26410 heartbeater.cc:503] Master 127.12.117.190:44969 was elected leader, sending a full tablet report...
I20260810 07:15:38.651593 26457 ts_tablet_manager.cc:1428] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260810 07:15:38.651707 26460 ts_tablet_manager.cc:1428] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:15:38.651927 26163 heartbeater.cc:503] Master 127.12.117.190:44969 was elected leader, sending a full tablet report...
I20260810 07:15:38.652478 26458 ts_tablet_manager.cc:1428] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260810 07:15:38.652504 25913 heartbeater.cc:503] Master 127.12.117.190:44969 was elected leader, sending a full tablet report...
W20260810 07:15:38.819496 26412 tablet.cc:2369] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:38.846485 26166 tablet.cc:2369] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:15:38.873631 25914 tablet.cc:2369] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:15:38.892359 26469 raft_consensus.cc:482] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:38.892542 26469 raft_consensus.cc:504] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.893659 26469 leader_election.cc:288] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875)
I20260810 07:15:38.897393 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.893924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44875 (local address 127.12.117.131:55559)
0810 07:15:38.894275 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.894286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.894378 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.894591 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.894592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.894604 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:38.894719 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.894724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.895507 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.895510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.896330 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.896340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.896975 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.896978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.897082 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.897101 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.897175 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.897244 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260810 07:15:38.897383 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.894024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:55559 (local address 127.12.117.130:44875)
0810 07:15:38.894139 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.894144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.894361 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:15:38.894454 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.894456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.894513 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.894578 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:38.894842 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.895399 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.896459 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.896863 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.896973 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.897030 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.897207 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.897280 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:15:38.897665 26228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.893885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.12.117.131:48749)
0810 07:15:38.894071 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:38.894085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:38.894197 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:38.894670 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:38.894672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:38.894688 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:38.895107 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.895113 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.895887 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.895891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.896622 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:38.896630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.897260 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.897264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:38.897361 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.897379 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:15:38.897442 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:15:38.897535 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 07:15:38.897778 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.893961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:48749 (local address 127.12.117.129:42253)
0810 07:15:38.894083 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.894088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.894182 (+ 94us) server_negotiation.cc:407] Connection header received
0810 07:15:38.894317 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.894319 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.894372 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.894437 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:38.895227 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.895777 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.896741 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.897150 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.897259 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.897313 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.897588 (+ 275us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.897672 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:15:38.898142 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66ac68f5d3b43ecbefdd12ce2821717" candidate_uuid: "96b9d45c053f4d4eaca06e24c17624a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24" is_pre_election: true
I20260810 07:15:38.898327 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66ac68f5d3b43ecbefdd12ce2821717" candidate_uuid: "96b9d45c053f4d4eaca06e24c17624a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" is_pre_election: true
I20260810 07:15:38.898483 26082 raft_consensus.cc:2454] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b9d45c053f4d4eaca06e24c17624a4 in term 0.
I20260810 07:15:38.898587 25824 raft_consensus.cc:2454] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96b9d45c053f4d4eaca06e24c17624a4 in term 0.
I20260810 07:15:38.898959 26210 leader_election.cc:302] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 96b9d45c053f4d4eaca06e24c17624a4; no voters:
I20260810 07:15:38.899209 26469 raft_consensus.cc:2781] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:38.899313 26469 raft_consensus.cc:482] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:38.899366 26469 raft_consensus.cc:3037] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:38.900368 26469 raft_consensus.cc:504] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.900678 26469 leader_election.cc:288] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [CANDIDATE]: Term 1 election: Requested vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875)
I20260810 07:15:38.900986 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66ac68f5d3b43ecbefdd12ce2821717" candidate_uuid: "96b9d45c053f4d4eaca06e24c17624a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
I20260810 07:15:38.901039 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f66ac68f5d3b43ecbefdd12ce2821717" candidate_uuid: "96b9d45c053f4d4eaca06e24c17624a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24"
I20260810 07:15:38.901103 25824 raft_consensus.cc:3037] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:38.901161 26082 raft_consensus.cc:3037] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:38.902161 25824 raft_consensus.cc:2454] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b9d45c053f4d4eaca06e24c17624a4 in term 1.
I20260810 07:15:38.902172 26082 raft_consensus.cc:2454] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96b9d45c053f4d4eaca06e24c17624a4 in term 1.
I20260810 07:15:38.902470 26210 leader_election.cc:302] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 96b9d45c053f4d4eaca06e24c17624a4; no voters:
I20260810 07:15:38.902652 26469 raft_consensus.cc:2781] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:38.902839 26469 raft_consensus.cc:686] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 LEADER]: Becoming Leader. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Running, Role: LEADER
I20260810 07:15:38.903136 26469 consensus_queue.cc:237] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.906129 25541 catalog_manager.cc:5495] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "96b9d45c053f4d4eaca06e24c17624a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } health_report { overall_health: HEALTHY } } }
I20260810 07:15:38.916985 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:38.913530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.12.117.131:36327)
0810 07:15:38.913641 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:38.913645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:38.913695 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:15:38.913854 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:38.913856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:38.913905 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:38.913990 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:38.914234 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.914721 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.915603 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:38.915821 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:38.915910 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:38.916246 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:38.916629 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:38.916729 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:38.916817 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 07:15:38.916870 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:15:38.922021 26082 raft_consensus.cc:1261] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Refusing update from remote peer 96b9d45c053f4d4eaca06e24c17624a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:38.922020 25824 raft_consensus.cc:1261] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Refusing update from remote peer 96b9d45c053f4d4eaca06e24c17624a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:38.922592 26469 consensus_queue.cc:1035] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:38.922806 26474 consensus_queue.cc:1035] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:38.930691 26485 mvcc.cc:205] Tried to move back new op lower bound from 7316873785018531840 to 7316873784949981184. Current Snapshot: MvccSnapshot[applied={T|T < 7316873785018531840}]
I20260810 07:15:38.938185 25541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:38.938333 25541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:38.940546 25541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Sending DeleteTablet for 3 replicas of tablet f66ac68f5d3b43ecbefdd12ce2821717
I20260810 07:15:38.941331 26295 tablet_service.cc:1514] Processing DeleteTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:38 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:41834
I20260810 07:15:38.941331 25789 tablet_service.cc:1514] Processing DeleteTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:38 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60362
I20260810 07:15:38.941824 26498 tablet_replica.cc:330] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: stopping tablet replica
I20260810 07:15:38.941805 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet f66ac68f5d3b43ecbefdd12ce2821717 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:38 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:42144
I20260810 07:15:38.942162 26498 raft_consensus.cc:2229] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:38.942194 26502 tablet_replica.cc:330] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: stopping tablet replica
I20260810 07:15:38.942466 26498 raft_consensus.cc:2258] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:38.942622 26502 raft_consensus.cc:2229] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:38.942809 26500 tablet_replica.cc:330] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: stopping tablet replica
I20260810 07:15:38.942773 25541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:38.943078 26500 raft_consensus.cc:2229] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 LEADER]: Raft consensus shutting down.
W20260810 07:15:38.943166 25541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:38.943148 26502 raft_consensus.cc:2258] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:38.943471 26500 raft_consensus.cc:2258] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:38.944056 26498 ts_tablet_manager.cc:1905] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:38.944777 26500 ts_tablet_manager.cc:1905] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:38.946904 26498 ts_tablet_manager.cc:1918] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:38.947024 26498 log.cc:1196] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/f66ac68f5d3b43ecbefdd12ce2821717
I20260810 07:15:38.947378 26498 ts_tablet_manager.cc:1939] T f66ac68f5d3b43ecbefdd12ce2821717 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting consensus metadata
I20260810 07:15:38.947405 26500 ts_tablet_manager.cc:1918] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:38.947481 26500 log.cc:1196] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/f66ac68f5d3b43ecbefdd12ce2821717
I20260810 07:15:38.947767 26500 ts_tablet_manager.cc:1939] T f66ac68f5d3b43ecbefdd12ce2821717 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting consensus metadata
I20260810 07:15:38.948009 26295 tablet_service.cc:1467] Processing CreateTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 (DEFAULT_TABLE table=test-table [id=23ffe066b11a4e47b25589d45815004a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.948145 26041 tablet_service.cc:1467] Processing CreateTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 (DEFAULT_TABLE table=test-table [id=23ffe066b11a4e47b25589d45815004a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.948253 26295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8189f4c8a6bb482cb6040a3d52e24af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.948393 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8189f4c8a6bb482cb6040a3d52e24af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.948575 25527 catalog_manager.cc:4841] TS 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253): tablet f66ac68f5d3b43ecbefdd12ce2821717 (table test-table [id=8d160f396fe34058866ca984b29cba40]) successfully deleted
I20260810 07:15:38.950350 26460 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Bootstrap starting.
I20260810 07:15:38.950692 26457 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Bootstrap starting.
I20260810 07:15:38.951021 25524 catalog_manager.cc:4841] TS 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327): tablet f66ac68f5d3b43ecbefdd12ce2821717 (table test-table [id=8d160f396fe34058866ca984b29cba40]) successfully deleted
I20260810 07:15:38.951370 26460 tablet_bootstrap.cc:654] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.951452 25789 tablet_service.cc:1467] Processing CreateTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 (DEFAULT_TABLE table=test-table [id=23ffe066b11a4e47b25589d45815004a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:38.951625 26457 tablet_bootstrap.cc:654] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.951684 25789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8189f4c8a6bb482cb6040a3d52e24af4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:38.952821 26457 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: No bootstrap required, opened a new log
I20260810 07:15:38.952917 26457 ts_tablet_manager.cc:1397] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:38.953363 26457 raft_consensus.cc:348] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.953482 26457 raft_consensus.cc:374] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.953526 26457 raft_consensus.cc:729] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.953647 26457 consensus_queue.cc:260] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.953830 26458 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Bootstrap starting.
I20260810 07:15:38.953907 26457 ts_tablet_manager.cc:1428] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:38.954764 26458 tablet_bootstrap.cc:654] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:38.955209 26460 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: No bootstrap required, opened a new log
I20260810 07:15:38.955304 26460 ts_tablet_manager.cc:1397] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260810 07:15:38.955734 26460 raft_consensus.cc:348] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.955844 26460 raft_consensus.cc:374] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.955878 26460 raft_consensus.cc:729] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.956014 26458 tablet_bootstrap.cc:492] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: No bootstrap required, opened a new log
I20260810 07:15:38.956008 26460 consensus_queue.cc:260] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.956084 26458 ts_tablet_manager.cc:1397] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 07:15:38.956250 26460 ts_tablet_manager.cc:1428] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:38.956635 26458 raft_consensus.cc:348] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.956737 26458 raft_consensus.cc:374] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:38.956768 26458 raft_consensus.cc:729] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:38.956878 26458 consensus_queue.cc:260] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:38.957118 26458 ts_tablet_manager.cc:1428] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 07:15:38.957425 26502 ts_tablet_manager.cc:1905] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:38.959890 26502 ts_tablet_manager.cc:1918] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:38.959968 26502 log.cc:1196] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/f66ac68f5d3b43ecbefdd12ce2821717
I20260810 07:15:38.960206 26502 ts_tablet_manager.cc:1939] T f66ac68f5d3b43ecbefdd12ce2821717 P 5aa6f289797f4ef096590b72c41d7b24: Deleting consensus metadata
I20260810 07:15:38.961099 25524 catalog_manager.cc:4841] TS 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875): tablet f66ac68f5d3b43ecbefdd12ce2821717 (table test-table [id=8d160f396fe34058866ca984b29cba40]) successfully deleted
I20260810 07:15:39.021086 26470 raft_consensus.cc:482] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:39.021204 26470 raft_consensus.cc:504] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.022210 26470 leader_election.cc:288] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.026788 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.022544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47871 (local address 127.12.117.130:44875)
0810 07:15:39.022661 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.022666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.022832 (+ 166us) server_negotiation.cc:407] Connection header received
0810 07:15:39.023004 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.023007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.023055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.023105 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.023552 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.024179 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.025550 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.026102 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.026257 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.026309 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.026610 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.026687 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 07:15:39.026818 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.022473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44875 (local address 127.12.117.129:47871)
0810 07:15:39.022752 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.022763 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.022866 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.023252 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.023255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.023303 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.023456 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.023463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.024284 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.024287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.025403 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.025415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.026290 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.026294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.026404 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.026465 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.026569 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.026648 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"spinlock_wait_cycles":26240,"thread_start_us":113,"threads_started":1}
I20260810 07:15:39.026945 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.023012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:41305 (local address 127.12.117.131:36327)
0810 07:15:39.023168 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.023173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.023325 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:15:39.023470 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.023473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.023520 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.023598 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.023838 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.024590 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.025882 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.026328 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.026436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.026487 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.026749 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.026827 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:15:39.027333 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.022862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:36327 (local address 127.12.117.129:41305)
0810 07:15:39.023227 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.023239 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.023363 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.023601 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.023604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.023634 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.023730 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.023735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.024713 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.024716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.025772 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.025780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.026484 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.026489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.026593 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.026619 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.027081 (+ 462us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.027171 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"spinlock_wait_cycles":384,"thread_start_us":94,"threads_started":1}
I20260810 07:15:39.027372 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8189f4c8a6bb482cb6040a3d52e24af4" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24" is_pre_election: true
I20260810 07:15:39.027518 26082 raft_consensus.cc:2454] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 0.
I20260810 07:15:39.027933 25701 leader_election.cc:302] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.027993 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8189f4c8a6bb482cb6040a3d52e24af4" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4" is_pre_election: true
I20260810 07:15:39.028187 26335 raft_consensus.cc:2454] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 0.
I20260810 07:15:39.028427 26470 raft_consensus.cc:2781] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:39.028496 26470 raft_consensus.cc:482] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:39.028539 26470 raft_consensus.cc:3037] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.029487 26470 raft_consensus.cc:504] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.029811 26470 leader_election.cc:288] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 election: Requested vote from peers 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.030123 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8189f4c8a6bb482cb6040a3d52e24af4" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24"
I20260810 07:15:39.030236 26082 raft_consensus.cc:3037] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.030252 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8189f4c8a6bb482cb6040a3d52e24af4" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4"
I20260810 07:15:39.030361 26335 raft_consensus.cc:3037] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.031155 26082 raft_consensus.cc:2454] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 1.
I20260810 07:15:39.031462 25701 leader_election.cc:302] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.031533 26335 raft_consensus.cc:2454] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 1.
I20260810 07:15:39.031623 26470 raft_consensus.cc:2781] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.031803 26470 raft_consensus.cc:686] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 LEADER]: Becoming Leader. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Running, Role: LEADER
I20260810 07:15:39.032049 26470 consensus_queue.cc:237] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.034195 25539 catalog_manager.cc:5495] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129). New cstate: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:15:39 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346137, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.190@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:39.085768 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.077021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35078 (local address 127.12.117.190:44969)
0810 07:15:39.077159 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.077164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.077346 (+ 182us) server_negotiation.cc:407] Connection header received
0810 07:15:39.077480 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.077482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.077524 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.077589 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.078429 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.078971 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.080101 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.080309 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.080380 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.082786 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.082789 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.082791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.082812 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.084188 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.084616 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.084620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.084622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.084693 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.084928 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.084931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.084932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.085110 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.085196 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.085477 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.085635 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":48}
I20260810 07:15:39.085768 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.076913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44969 (local address 127.0.0.1:35078)
0810 07:15:39.077242 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.077257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.077399 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.077625 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.077628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.078136 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:39.078319 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.078328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.079098 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.079101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.079981 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.079994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.080415 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.080419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.080509 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.081044 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:39.081058 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:39.082661 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:39.084281 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.084291 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.084297 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.084529 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.084771 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:39.084774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:39.084776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:39.084852 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:39.085208 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:39.085215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:39.085320 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.085417 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.085583 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":105,"threads_started":1}
I20260810 07:15:39.089324 25539 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:35078:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 07:15:39.090025 25539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:39.094743 25789 tablet_service.cc:1467] Processing CreateTablet for tablet e5e2083d60274ee2abb6cf35af474cc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53139ceb74fb4ba1ba3cf2a527ef50ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:39.094763 26295 tablet_service.cc:1467] Processing CreateTablet for tablet e5e2083d60274ee2abb6cf35af474cc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53139ceb74fb4ba1ba3cf2a527ef50ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:39.094780 26041 tablet_service.cc:1467] Processing CreateTablet for tablet e5e2083d60274ee2abb6cf35af474cc5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=53139ceb74fb4ba1ba3cf2a527ef50ea]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 07:15:39.095131 26295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e2083d60274ee2abb6cf35af474cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.095131 25789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e2083d60274ee2abb6cf35af474cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.095170 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e2083d60274ee2abb6cf35af474cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.097669 26457 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: Bootstrap starting.
I20260810 07:15:39.097827 26460 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: Bootstrap starting.
I20260810 07:15:39.098266 26458 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: Bootstrap starting.
I20260810 07:15:39.098663 26457 tablet_bootstrap.cc:654] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.098680 26460 tablet_bootstrap.cc:654] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.099195 26458 tablet_bootstrap.cc:654] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.099874 26460 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: No bootstrap required, opened a new log
I20260810 07:15:39.099956 26460 ts_tablet_manager.cc:1397] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:39.100383 26457 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: No bootstrap required, opened a new log
I20260810 07:15:39.100391 26458 tablet_bootstrap.cc:492] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: No bootstrap required, opened a new log
I20260810 07:15:39.100363 26460 raft_consensus.cc:348] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.100454 26457 ts_tablet_manager.cc:1397] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:15:39.100469 26458 ts_tablet_manager.cc:1397] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:39.100490 26460 raft_consensus.cc:374] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.100521 26460 raft_consensus.cc:729] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.100646 26460 consensus_queue.cc:260] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.100849 26458 raft_consensus.cc:348] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.100895 26460 ts_tablet_manager.cc:1428] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:39.100879 26457 raft_consensus.cc:348] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.100955 26458 raft_consensus.cc:374] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.100878 26489 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.100986 26458 raft_consensus.cc:729] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.100987 26457 raft_consensus.cc:374] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.101019 26457 raft_consensus.cc:729] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.101030 26489 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.101104 26458 consensus_queue.cc:260] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.101137 26457 consensus_queue.cc:260] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.101374 26457 ts_tablet_manager.cc:1428] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:39.101344 26470 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.101487 26470 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.101481 26468 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.101634 26468 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.101351 26458 ts_tablet_manager.cc:1428] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.243589 26470 raft_consensus.cc:482] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:39.243716 26470 raft_consensus.cc:504] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.244078 26470 leader_election.cc:288] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.244509 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e2083d60274ee2abb6cf35af474cc5" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4" is_pre_election: true
I20260810 07:15:39.244513 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e2083d60274ee2abb6cf35af474cc5" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24" is_pre_election: true
I20260810 07:15:39.244648 26082 raft_consensus.cc:2454] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 0.
I20260810 07:15:39.244655 26335 raft_consensus.cc:2454] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 0.
I20260810 07:15:39.244937 25701 leader_election.cc:302] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.245126 26470 raft_consensus.cc:2781] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:39.245185 26470 raft_consensus.cc:482] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:39.245229 26470 raft_consensus.cc:3037] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.246150 26470 raft_consensus.cc:504] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.246469 26470 leader_election.cc:288] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 election: Requested vote from peers 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.246822 26082 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e2083d60274ee2abb6cf35af474cc5" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5aa6f289797f4ef096590b72c41d7b24"
I20260810 07:15:39.246821 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e2083d60274ee2abb6cf35af474cc5" candidate_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4"
I20260810 07:15:39.246953 26082 raft_consensus.cc:3037] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.246963 26335 raft_consensus.cc:3037] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.247861 26082 raft_consensus.cc:2454] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 1.
I20260810 07:15:39.247947 26335 raft_consensus.cc:2454] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ec3d8f7e3d4d848905098b89c6b1ad in term 1.
I20260810 07:15:39.248124 25701 leader_election.cc:302] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.248281 26470 raft_consensus.cc:2781] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.248368 26470 raft_consensus.cc:686] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 LEADER]: Becoming Leader. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Running, Role: LEADER
I20260810 07:15:39.248500 26470 consensus_queue.cc:237] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.249245 26509 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: TxnStatusTablet state changed. Reason: New leader 94ec3d8f7e3d4d848905098b89c6b1ad. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.249378 26509 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:39.249655 26527 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 07:15:39.249856 26527 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 07:15:39.250229 25540 catalog_manager.cc:5495] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129). New cstate: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:39.292305 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.288711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.12.117.129:42253)
0810 07:15:39.288824 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.288828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.288878 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:15:39.289074 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.289077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.289131 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.289213 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.289452 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.289964 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.290835 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.291071 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.291162 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.291505 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.291899 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.292011 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.292117 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.292171 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:15:39.292305 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.288621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.0.0.1:60370)
0810 07:15:39.288792 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.288809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.288938 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.289239 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.289242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.289250 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:39.289350 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.289355 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.290077 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.290080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.290694 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.290703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.291178 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.291182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.291304 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.291357 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.292024 (+ 667us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.292042 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.292105 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.292192 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 07:15:39.295416 26082 raft_consensus.cc:1261] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Refusing update from remote peer 94ec3d8f7e3d4d848905098b89c6b1ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.295630 26335 raft_consensus.cc:1261] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Refusing update from remote peer 94ec3d8f7e3d4d848905098b89c6b1ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.296190 26470 consensus_queue.cc:1035] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.296370 26509 consensus_queue.cc:1035] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.298009 26468 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: TxnStatusTablet state changed. Reason: New leader 94ec3d8f7e3d4d848905098b89c6b1ad. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.298130 26468 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.298060 26489 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: TxnStatusTablet state changed. Reason: New leader 94ec3d8f7e3d4d848905098b89c6b1ad. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.298172 26489 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.299856 26509 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.299957 26509 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:39.300379 26468 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.300478 26468 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 5aa6f289797f4ef096590b72c41d7b24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.300591 26486 mvcc.cc:205] Tried to move back new op lower bound from 7316873786549166080 to 7316873786363379712. Current Snapshot: MvccSnapshot[applied={T|T < 7316873786549166080 or (T in {7316873786549166080})}]
I20260810 07:15:39.301124 26470 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.301225 26470 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 94ec3d8f7e3d4d848905098b89c6b1ad: This TxnStatusTablet replica's current role is: LEADER
I20260810 07:15:39.301878 26489 tablet_replica.cc:439] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } } }
I20260810 07:15:39.301992 26489 tablet_replica.cc:442] T e5e2083d60274ee2abb6cf35af474cc5 P 96b9d45c053f4d4eaca06e24c17624a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 07:15:39.309948 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.306881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60374 (local address 127.12.117.129:42253)
0810 07:15:39.306979 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.306983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.307061 (+ 78us) server_negotiation.cc:407] Connection header received
0810 07:15:39.307190 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.307194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.307236 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.307335 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.307568 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.308059 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.308927 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.309155 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.309245 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.309518 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.309595 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.309654 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.309770 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.309822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:39.320062 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.316629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59535 (local address 127.12.117.129:42253)
0810 07:15:39.316741 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.316745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.316907 (+ 162us) server_negotiation.cc:407] Connection header received
0810 07:15:39.317019 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.317021 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.317058 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.317114 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.317370 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.317826 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.319011 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.319204 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.319300 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.319657 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.319716 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.319794 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.319880 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.319933 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":45}
I20260810 07:15:39.320132 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.316552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.12.117.129:59535)
0810 07:15:39.316829 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.316843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.316943 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.317139 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.317142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.317149 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:39.317274 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.317281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.317911 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.317914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.318907 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.318918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.319333 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.319337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.319463 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.319524 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.319804 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.319810 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.319873 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.319953 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260810 07:15:39.326812 26335 raft_consensus.cc:1261] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Refusing update from remote peer 94ec3d8f7e3d4d848905098b89c6b1ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.326859 26082 raft_consensus.cc:1261] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Refusing update from remote peer 94ec3d8f7e3d4d848905098b89c6b1ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.327258 26470 consensus_queue.cc:1035] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.327438 26532 consensus_queue.cc:1035] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.332454 26482 mvcc.cc:205] Tried to move back new op lower bound from 7316873786679455744 to 7316873785477349376. Current Snapshot: MvccSnapshot[applied={T|T < 7316873786679455744}]
I20260810 07:15:39.381480 25541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.381665 25541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.383667 25541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Sending DeleteTablet for 3 replicas of tablet 8189f4c8a6bb482cb6040a3d52e24af4
I20260810 07:15:39.384160 25787 tablet_service.cc:1514] Processing DeleteTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60362
I20260810 07:15:39.384342 26498 tablet_replica.cc:330] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: stopping tablet replica
I20260810 07:15:39.384311 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:42144
I20260810 07:15:39.384454 26498 raft_consensus.cc:2229] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:39.384493 26502 tablet_replica.cc:330] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: stopping tablet replica
I20260810 07:15:39.384603 26502 raft_consensus.cc:2229] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.384629 26295 tablet_service.cc:1514] Processing DeleteTablet for tablet 8189f4c8a6bb482cb6040a3d52e24af4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:41834
I20260810 07:15:39.384707 26502 raft_consensus.cc:2258] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.384730 26498 raft_consensus.cc:2258] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.384790 26500 tablet_replica.cc:330] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: stopping tablet replica
I20260810 07:15:39.384891 26500 raft_consensus.cc:2229] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.384980 26500 raft_consensus.cc:2258] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.385797 26498 ts_tablet_manager.cc:1905] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.388863 26498 ts_tablet_manager.cc:1918] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:39.388947 26498 log.cc:1196] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/8189f4c8a6bb482cb6040a3d52e24af4
I20260810 07:15:39.389195 26498 ts_tablet_manager.cc:1939] T 8189f4c8a6bb482cb6040a3d52e24af4 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting consensus metadata
I20260810 07:15:39.389900 25527 catalog_manager.cc:4841] TS 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253): tablet 8189f4c8a6bb482cb6040a3d52e24af4 (table test-table [id=23ffe066b11a4e47b25589d45815004a]) successfully deleted
I20260810 07:15:39.390112 26502 ts_tablet_manager.cc:1905] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.392699 26502 ts_tablet_manager.cc:1918] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:39.392824 26502 log.cc:1196] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/8189f4c8a6bb482cb6040a3d52e24af4
I20260810 07:15:39.393100 26502 ts_tablet_manager.cc:1939] T 8189f4c8a6bb482cb6040a3d52e24af4 P 5aa6f289797f4ef096590b72c41d7b24: Deleting consensus metadata
I20260810 07:15:39.393747 25524 catalog_manager.cc:4841] TS 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875): tablet 8189f4c8a6bb482cb6040a3d52e24af4 (table test-table [id=23ffe066b11a4e47b25589d45815004a]) successfully deleted
I20260810 07:15:39.394449 26500 ts_tablet_manager.cc:1905] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.397260 26500 ts_tablet_manager.cc:1918] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:39.397344 26500 log.cc:1196] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/8189f4c8a6bb482cb6040a3d52e24af4
I20260810 07:15:39.397584 26500 ts_tablet_manager.cc:1939] T 8189f4c8a6bb482cb6040a3d52e24af4 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting consensus metadata
Aug 10 07:15:39 dist-test-slave-gjrz krb5kdc[25473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260810 07:15:39.398419 25524 catalog_manager.cc:4841] TS 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327): tablet 8189f4c8a6bb482cb6040a3d52e24af4 (table test-table [id=23ffe066b11a4e47b25589d45815004a]) successfully deleted
Aug 10 07:15:39 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:39 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:39.410003 25638 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:39.403939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.12.117.190:44969)
0810 07:15:39.404052 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.404056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.404249 (+ 193us) server_negotiation.cc:407] Connection header received
0810 07:15:39.404374 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.404377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.404424 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.404489 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.405145 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.405667 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.406489 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.406716 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.406803 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.409899 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35094: BlockingRecv error: recv got EOF from 127.0.0.1:35094 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:15:39 dist-test-slave-gjrz krb5kdc[25473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM
I20260810 07:15:39.419190 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.411347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.12.117.190:44969)
0810 07:15:39.411450 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.411454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.411553 (+ 99us) server_negotiation.cc:407] Connection header received
0810 07:15:39.411729 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.411733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.411776 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.411816 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:39.412419 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.412900 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.413755 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.413980 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.414066 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.416417 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:39.416420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:39.416423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:39.416446 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:39.417715 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.418152 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.418154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.418155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.418223 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:39.418464 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:39.418466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:39.418467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:39.418617 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:39.418708 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.418926 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.419059 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:39.421419 25541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:15:39.421811 25541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:39.426182 25787 tablet_service.cc:1467] Processing CreateTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 (DEFAULT_TABLE table=test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.426380 26041 tablet_service.cc:1467] Processing CreateTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 (DEFAULT_TABLE table=test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.426504 25787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e3a75cbb474a349bd38dafb43cd9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.426515 26295 tablet_service.cc:1467] Processing CreateTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 (DEFAULT_TABLE table=test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.426621 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e3a75cbb474a349bd38dafb43cd9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.426783 26295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e3a75cbb474a349bd38dafb43cd9b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.428928 26458 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Bootstrap starting.
I20260810 07:15:39.428939 26457 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Bootstrap starting.
I20260810 07:15:39.429062 26460 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Bootstrap starting.
I20260810 07:15:39.429736 26457 tablet_bootstrap.cc:654] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.429736 26458 tablet_bootstrap.cc:654] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.429837 26460 tablet_bootstrap.cc:654] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.431195 26457 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: No bootstrap required, opened a new log
I20260810 07:15:39.431300 26457 ts_tablet_manager.cc:1397] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:39.431454 26460 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: No bootstrap required, opened a new log
I20260810 07:15:39.431537 26460 ts_tablet_manager.cc:1397] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260810 07:15:39.431901 26460 raft_consensus.cc:348] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.432014 26460 raft_consensus.cc:374] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.431958 26457 raft_consensus.cc:348] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.432044 26460 raft_consensus.cc:729] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.432051 26457 raft_consensus.cc:374] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.432085 26457 raft_consensus.cc:729] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.432157 26460 consensus_queue.cc:260] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.432194 26457 consensus_queue.cc:260] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.432399 26460 ts_tablet_manager.cc:1428] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.432441 26457 ts_tablet_manager.cc:1428] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.433748 26458 tablet_bootstrap.cc:492] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: No bootstrap required, opened a new log
I20260810 07:15:39.433835 26458 ts_tablet_manager.cc:1397] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260810 07:15:39.434231 26458 raft_consensus.cc:348] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.434342 26458 raft_consensus.cc:374] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.434372 26458 raft_consensus.cc:729] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.434490 26458 consensus_queue.cc:260] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.434732 26458 ts_tablet_manager.cc:1428] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.680481 26468 raft_consensus.cc:482] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:39.680615 26468 raft_consensus.cc:504] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.681649 26468 leader_election.cc:288] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.686803 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.681947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:57663 (local address 127.12.117.129:42253)
0810 07:15:39.682083 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.682088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.682302 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:15:39.682492 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.682496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.682549 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.682640 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.682975 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.683793 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.685580 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.686193 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.686325 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.686388 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.686598 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.686672 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:15:39.686806 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.682033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:34689 (local address 127.12.117.131:36327)
0810 07:15:39.682221 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.682225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.682449 (+ 224us) server_negotiation.cc:407] Connection header received
0810 07:15:39.682507 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.682510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.682556 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.682640 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:15:39.682975 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.683793 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.685580 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.686236 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.686341 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.686388 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.686600 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.686681 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":55,"threads_started":1}
I20260810 07:15:39.686861 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.681843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.12.117.130:57663)
0810 07:15:39.682202 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.682217 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.682335 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.682655 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.682658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.682679 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.682857 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.682865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.683905 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.683909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.685431 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.685445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.686309 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.686312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.686450 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.686495 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.686572 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.686674 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":179,"thread_start_us":120,"threads_started":1}
I20260810 07:15:39.687286 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.681903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:36327 (local address 127.12.117.130:34689)
0810 07:15:39.682335 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.682348 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.682428 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.682655 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.682658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.682688 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:15:39.682857 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.682865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.683905 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.683909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.685431 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.685445 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.686328 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.686331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.686458 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.686502 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.686572 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.686672 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":203,"spinlock_wait_cycles":8448,"thread_start_us":66,"threads_started":1}
I20260810 07:15:39.687294 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e3a75cbb474a349bd38dafb43cd9b3" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" is_pre_election: true
I20260810 07:15:39.687436 25824 raft_consensus.cc:2454] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 0.
I20260810 07:15:39.687702 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e3a75cbb474a349bd38dafb43cd9b3" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4" is_pre_election: true
I20260810 07:15:39.687840 26335 raft_consensus.cc:2454] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 0.
I20260810 07:15:39.687793 25960 leader_election.cc:302] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.688134 26468 raft_consensus.cc:2781] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:39.688205 26468 raft_consensus.cc:482] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:39.688242 26468 raft_consensus.cc:3037] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.689224 26468 raft_consensus.cc:504] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.689565 26468 leader_election.cc:288] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 election: Requested vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.689913 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e3a75cbb474a349bd38dafb43cd9b3" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
I20260810 07:15:39.689905 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e3a75cbb474a349bd38dafb43cd9b3" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4"
I20260810 07:15:39.690037 26335 raft_consensus.cc:3037] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.690037 25824 raft_consensus.cc:3037] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.691184 25824 raft_consensus.cc:2454] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 1.
I20260810 07:15:39.691184 26335 raft_consensus.cc:2454] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 1.
I20260810 07:15:39.691520 25953 leader_election.cc:302] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 96b9d45c053f4d4eaca06e24c17624a4; no voters:
I20260810 07:15:39.691694 26468 raft_consensus.cc:2781] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.691851 26468 raft_consensus.cc:686] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 LEADER]: Becoming Leader. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Running, Role: LEADER
I20260810 07:15:39.692094 26468 consensus_queue.cc:237] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.694180 25541 catalog_manager.cc:5495] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130). New cstate: current_term: 1 leader_uuid: "5aa6f289797f4ef096590b72c41d7b24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:39.733942 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.730259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.12.117.130:44875)
0810 07:15:39.730530 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.730536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.730547 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:15:39.730621 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.730623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.730673 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.730777 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.731019 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.731585 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.732429 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.732722 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.732813 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.733091 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.733532 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.733648 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.733750 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.733800 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":96,"threads_started":1}
I20260810 07:15:39.736246 26335 raft_consensus.cc:1261] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Refusing update from remote peer 5aa6f289797f4ef096590b72c41d7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.736272 25824 raft_consensus.cc:1261] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Refusing update from remote peer 5aa6f289797f4ef096590b72c41d7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.736717 26568 consensus_queue.cc:1035] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.736892 26468 consensus_queue.cc:1035] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.747165 25540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.747332 25540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.749316 25540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Sending DeleteTablet for 3 replicas of tablet 83e3a75cbb474a349bd38dafb43cd9b3
I20260810 07:15:39.749800 25787 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60362
I20260810 07:15:39.749922 26295 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:41834
I20260810 07:15:39.749950 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e3a75cbb474a349bd38dafb43cd9b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:42144
I20260810 07:15:39.750006 26498 tablet_replica.cc:330] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: stopping tablet replica
I20260810 07:15:39.750072 26500 tablet_replica.cc:330] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: stopping tablet replica
I20260810 07:15:39.750118 26498 raft_consensus.cc:2229] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.750126 26502 tablet_replica.cc:330] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: stopping tablet replica
I20260810 07:15:39.750164 26500 raft_consensus.cc:2229] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.750231 26498 raft_consensus.cc:2258] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.750231 26502 raft_consensus.cc:2229] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:39.750262 26500 raft_consensus.cc:2258] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.750486 26502 raft_consensus.cc:2258] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.751165 25540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 07:15:39.751295 26500 ts_tablet_manager.cc:1905] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.751569 26502 ts_tablet_manager.cc:1905] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:15:39.751803 25540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:15:39.753636 26502 ts_tablet_manager.cc:1918] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:39.753703 26502 log.cc:1196] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/83e3a75cbb474a349bd38dafb43cd9b3
I20260810 07:15:39.753821 26500 ts_tablet_manager.cc:1918] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:39.753906 26500 log.cc:1196] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/83e3a75cbb474a349bd38dafb43cd9b3
I20260810 07:15:39.753968 26502 ts_tablet_manager.cc:1939] T 83e3a75cbb474a349bd38dafb43cd9b3 P 5aa6f289797f4ef096590b72c41d7b24: Deleting consensus metadata
I20260810 07:15:39.754156 26500 ts_tablet_manager.cc:1939] T 83e3a75cbb474a349bd38dafb43cd9b3 P 96b9d45c053f4d4eaca06e24c17624a4: Deleting consensus metadata
I20260810 07:15:39.754573 25524 catalog_manager.cc:4841] TS 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875): tablet 83e3a75cbb474a349bd38dafb43cd9b3 (table test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]) successfully deleted
I20260810 07:15:39.754810 25524 catalog_manager.cc:4841] TS 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327): tablet 83e3a75cbb474a349bd38dafb43cd9b3 (table test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]) successfully deleted
I20260810 07:15:39.755471 26498 ts_tablet_manager.cc:1905] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.757771 26498 ts_tablet_manager.cc:1918] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:15:39.757843 26498 log.cc:1196] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/83e3a75cbb474a349bd38dafb43cd9b3
I20260810 07:15:39.758010 25787 tablet_service.cc:1467] Processing CreateTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e (DEFAULT_TABLE table=test-table [id=38dafee27bd34f70b453eb3acda6f249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.758265 25787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832fd03e5c4c447c9133a3c08be5df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.758584 26295 tablet_service.cc:1467] Processing CreateTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e (DEFAULT_TABLE table=test-table [id=38dafee27bd34f70b453eb3acda6f249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.758838 26295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832fd03e5c4c447c9133a3c08be5df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.759524 26041 tablet_service.cc:1467] Processing CreateTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e (DEFAULT_TABLE table=test-table [id=38dafee27bd34f70b453eb3acda6f249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:15:39.759740 26041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832fd03e5c4c447c9133a3c08be5df8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:39.760341 26458 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: Bootstrap starting.
I20260810 07:15:39.760942 26460 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Bootstrap starting.
I20260810 07:15:39.761109 26458 tablet_bootstrap.cc:654] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.761711 26460 tablet_bootstrap.cc:654] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.761785 26457 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Bootstrap starting.
I20260810 07:15:39.762655 26457 tablet_bootstrap.cc:654] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:39.762990 26458 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: No bootstrap required, opened a new log
I20260810 07:15:39.763064 26458 ts_tablet_manager.cc:1397] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 07:15:39.758068 26498 ts_tablet_manager.cc:1939] T 83e3a75cbb474a349bd38dafb43cd9b3 P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting consensus metadata
I20260810 07:15:39.763484 26458 raft_consensus.cc:348] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.763581 26458 raft_consensus.cc:374] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.763617 26458 raft_consensus.cc:729] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ec3d8f7e3d4d848905098b89c6b1ad, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.763697 26457 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: No bootstrap required, opened a new log
I20260810 07:15:39.763721 26458 consensus_queue.cc:260] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.763770 26457 ts_tablet_manager.cc:1397] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:39.763888 25527 catalog_manager.cc:4841] TS 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253): tablet 83e3a75cbb474a349bd38dafb43cd9b3 (table test-table [id=0e049ae6ee3149aab0fb0434af2d39fa]) successfully deleted
I20260810 07:15:39.763949 26458 ts_tablet_manager.cc:1428] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.764180 26457 raft_consensus.cc:348] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.764290 26457 raft_consensus.cc:374] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.764325 26457 raft_consensus.cc:729] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.764447 26457 consensus_queue.cc:260] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.764706 26457 ts_tablet_manager.cc:1428] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:15:39.764942 26460 tablet_bootstrap.cc:492] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: No bootstrap required, opened a new log
I20260810 07:15:39.765014 26460 ts_tablet_manager.cc:1397] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 07:15:39.765331 26460 raft_consensus.cc:348] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.765412 26460 raft_consensus.cc:374] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:39.765432 26460 raft_consensus.cc:729] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96b9d45c053f4d4eaca06e24c17624a4, State: Initialized, Role: FOLLOWER
I20260810 07:15:39.765528 26460 consensus_queue.cc:260] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.765731 26460 ts_tablet_manager.cc:1428] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 07:15:39.866385 26578 raft_consensus.cc:482] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:15:39.866510 26578 raft_consensus.cc:504] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.866993 26578 leader_election.cc:288] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.867455 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832fd03e5c4c447c9133a3c08be5df8e" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" is_pre_election: true
I20260810 07:15:39.867456 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832fd03e5c4c447c9133a3c08be5df8e" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4" is_pre_election: true
I20260810 07:15:39.867600 26335 raft_consensus.cc:2454] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 0.
I20260810 07:15:39.867600 25824 raft_consensus.cc:2454] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 0.
I20260810 07:15:39.867920 25953 leader_election.cc:302] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 96b9d45c053f4d4eaca06e24c17624a4; no voters:
I20260810 07:15:39.868139 26578 raft_consensus.cc:2781] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:15:39.868193 26578 raft_consensus.cc:482] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:15:39.868228 26578 raft_consensus.cc:3037] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.869455 26578 raft_consensus.cc:504] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.869801 26578 leader_election.cc:288] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 election: Requested vote from peers 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253), 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327)
I20260810 07:15:39.870146 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832fd03e5c4c447c9133a3c08be5df8e" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad"
I20260810 07:15:39.870177 26335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832fd03e5c4c447c9133a3c08be5df8e" candidate_uuid: "5aa6f289797f4ef096590b72c41d7b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96b9d45c053f4d4eaca06e24c17624a4"
I20260810 07:15:39.870263 25824 raft_consensus.cc:3037] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.870263 26335 raft_consensus.cc:3037] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:39.871389 25824 raft_consensus.cc:2454] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 1.
I20260810 07:15:39.871592 26335 raft_consensus.cc:2454] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5aa6f289797f4ef096590b72c41d7b24 in term 1.
I20260810 07:15:39.871788 25960 leader_election.cc:302] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5aa6f289797f4ef096590b72c41d7b24, 94ec3d8f7e3d4d848905098b89c6b1ad; no voters:
I20260810 07:15:39.872007 26578 raft_consensus.cc:2781] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:39.872118 26578 raft_consensus.cc:686] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 1 LEADER]: Becoming Leader. State: Replica: 5aa6f289797f4ef096590b72c41d7b24, State: Running, Role: LEADER
I20260810 07:15:39.872262 26578 consensus_queue.cc:237] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } }
I20260810 07:15:39.873960 25540 catalog_manager.cc:5495] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130). New cstate: current_term: 1 leader_uuid: "5aa6f289797f4ef096590b72c41d7b24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5aa6f289797f4ef096590b72c41d7b24" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 } health_report { overall_health: UNKNOWN } } }
I20260810 07:15:39.900655 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.897807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:54785 (local address 127.12.117.129:42253)
0810 07:15:39.897920 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.897924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.898037 (+ 113us) server_negotiation.cc:407] Connection header received
0810 07:15:39.898141 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.898143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.898184 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.898248 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.898448 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.898939 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.899829 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.900038 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.900123 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.900325 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.900386 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.900431 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.900513 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.900552 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:39.900673 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.897711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.12.117.130:54785)
0810 07:15:39.897969 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.897982 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.898068 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.898274 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.898276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.898283 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:39.898371 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.898376 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.899031 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.899034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.899732 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.899743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.900120 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.900122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.900189 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.900232 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.900448 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.900453 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.900501 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.900567 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":73,"threads_started":1}
I20260810 07:15:39.905947 25824 raft_consensus.cc:1261] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Refusing update from remote peer 5aa6f289797f4ef096590b72c41d7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.906280 26335 raft_consensus.cc:1261] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Refusing update from remote peer 5aa6f289797f4ef096590b72c41d7b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:15:39.906297 26578 consensus_queue.cc:1035] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ec3d8f7e3d4d848905098b89c6b1ad" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 42253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.906611 26568 consensus_queue.cc:1035] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96b9d45c053f4d4eaca06e24c17624a4" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:15:39.921351 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.918451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:57879 (local address 127.12.117.130:44875)
0810 07:15:39.918557 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:39.918561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:39.918731 (+ 170us) server_negotiation.cc:407] Connection header received
0810 07:15:39.918829 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:39.918832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:39.918874 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:39.918932 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:15:39.919084 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.919620 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.920403 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.920655 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.920735 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.920964 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.921032 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.921089 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:39.921185 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:15:39.921237 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:15:39.921510 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.918396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44875 (local address 127.12.117.129:57879)
0810 07:15:39.918654 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.918668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:39.918760 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:39.918941 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:39.918943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:39.918949 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 07:15:39.919025 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.919030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.919701 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.919704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.920300 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:39.920310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:39.920749 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:39.920753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:39.920820 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:39.920848 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 07:15:39.921106 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 07:15:39.921112 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:15:39.921168 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 07:15:39.921373 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260810 07:15:39.939858 25540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.939982 25540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:15:39.941736 25540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b19119f74604b7b8d010a5481a2133a: Sending DeleteTablet for 3 replicas of tablet 832fd03e5c4c447c9133a3c08be5df8e
I20260810 07:15:39.942209 25787 tablet_service.cc:1514] Processing DeleteTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60362
I20260810 07:15:39.942260 26041 tablet_service.cc:1514] Processing DeleteTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:42144
I20260810 07:15:39.942320 26295 tablet_service.cc:1514] Processing DeleteTablet for tablet 832fd03e5c4c447c9133a3c08be5df8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:15:39 UTC) from {username='oryx', principal='oryx/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:41834
I20260810 07:15:39.942456 26502 tablet_replica.cc:330] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: stopping tablet replica
I20260810 07:15:39.942495 26498 tablet_replica.cc:330] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: stopping tablet replica
I20260810 07:15:39.942572 26502 raft_consensus.cc:2229] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:15:39.942613 26498 raft_consensus.cc:2229] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.942713 26498 raft_consensus.cc:2258] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.942739 26502 raft_consensus.cc:2258] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.942869 26500 tablet_replica.cc:330] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: stopping tablet replica
I20260810 07:15:39.942998 26500 raft_consensus.cc:2229] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:15:39.943104 26500 raft_consensus.cc:2258] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:15:39.943339 26498 ts_tablet_manager.cc:1905] T 832fd03e5c4c447c9133a3c08be5df8e P 94ec3d8f7e3d4d848905098b89c6b1ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.943974 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 25680
I20260810 07:15:39.945354 26502 ts_tablet_manager.cc:1905] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.947511 26502 ts_tablet_manager.cc:1918] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:39.947574 26502 log.cc:1196] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/832fd03e5c4c447c9133a3c08be5df8e
I20260810 07:15:39.947763 26502 ts_tablet_manager.cc:1939] T 832fd03e5c4c447c9133a3c08be5df8e P 5aa6f289797f4ef096590b72c41d7b24: Deleting consensus metadata
I20260810 07:15:39.952883 26500 ts_tablet_manager.cc:1905] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:15:39.955199 26500 ts_tablet_manager.cc:1918] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 07:15:39.957122 26500 log.cc:1196] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/832fd03e5c4c447c9133a3c08be5df8e
I20260810 07:15:39.957258 25524 catalog_manager.cc:4841] TS 5aa6f289797f4ef096590b72c41d7b24 (127.12.117.130:44875): tablet 832fd03e5c4c447c9133a3c08be5df8e (table test-table [id=38dafee27bd34f70b453eb3acda6f249]) successfully deleted
I20260810 07:15:39.957588 26500 ts_tablet_manager.cc:1939] T 832fd03e5c4c447c9133a3c08be5df8e P 96b9d45c053f4d4eaca06e24c17624a4: Deleting consensus metadata
I20260810 07:15:39.958168 25524 catalog_manager.cc:4841] TS 96b9d45c053f4d4eaca06e24c17624a4 (127.12.117.131:36327): tablet 832fd03e5c4c447c9133a3c08be5df8e (table test-table [id=38dafee27bd34f70b453eb3acda6f249]) successfully deleted
W20260810 07:15:39.965938 25527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:42253 (error 108)
I20260810 07:15:39.966161 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 25929
I20260810 07:15:39.976364 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:39.970325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:42253 (local address 127.0.0.1:35692)
0810 07:15:39.976136 (+ 5811us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:39.976247 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:42253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5706,"thread_start_us":964,"threads_started":1}
W20260810 07:15:39.978170 25527 catalog_manager.cc:4584] TS 94ec3d8f7e3d4d848905098b89c6b1ad (127.12.117.129:42253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 832fd03e5c4c447c9133a3c08be5df8e: Network error: Client connection negotiation failed: client connection to 127.12.117.129:42253: connect: Connection refused (error 111)
I20260810 07:15:39.984120 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 26184
I20260810 07:15:39.994186 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 25500
2026-08-10T07:15:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3595 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:40 dist-test-slave-gjrz krb5kdc[26610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:40 dist-test-slave-gjrz krb5kdc[26610](info): set up 2 sockets
Aug 10 07:15:40 dist-test-slave-gjrz krb5kdc[26610](info): commencing operation
krb5kdc: starting...
W20260810 07:15:41.175326 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:41 dist-test-slave-gjrz krb5kdc[26610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:41.286057 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44191
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:34511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:41.391731 26632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:41.391963 26632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:41.392016 26632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:41.395869 26632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:41.395917 26632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:41.395951 26632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:41.395980 26632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:41.396008 26632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:41.400931 26632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:34511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:41.401773 26632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:41.402647 26632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:41.410391 26644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:41.410406 26641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:41.410399 26648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:41.410480 26632 server_base.cc:1034] running on GCE node
I20260810 07:15:41.411099 26632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:41.412074 26632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:41.413225 26632 hybrid_clock.cc:648] HybridClock initialized: now 1786346141413214 us; error 23 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T07:15:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1423 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:15:41 dist-test-slave-gjrz krb5kdc[26655](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:15:41 dist-test-slave-gjrz krb5kdc[26655](info): set up 2 sockets
Aug 10 07:15:41 dist-test-slave-gjrz krb5kdc[26655](info): commencing operation
krb5kdc: starting...
W20260810 07:15:42.580041 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:15:42 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:15:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:15:42.691835 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42513
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:42513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:42.797786 26675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:42.798008 26675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:42.798044 26675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:42.801846 26675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:42.801903 26675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:42.801921 26675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:42.801936 26675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:42.801952 26675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:42.806841 26675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:42513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:42.807694 26675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:42.808552 26675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:42.816372 26688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:42.816483 26685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:42.816504 26675 server_base.cc:1034] running on GCE node
W20260810 07:15:42.816395 26690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:42.817168 26675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:42.818145 26675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:42.819322 26675 hybrid_clock.cc:648] HybridClock initialized: now 1786346142819306 us; error 30 us; skew 500 ppm
Aug 10 07:15:42 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346142, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:42.821805 26675 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:42.822640 26675 webserver.cc:460] Webserver started at http://127.12.117.190:42661/ using document root <none> and password file <none>
I20260810 07:15:42.823149 26675 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:42.823215 26675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:42.823432 26675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:42.825328 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "bf41c8a59fa048479959412135efcbce"
format_stamp: "Formatted at 2026-08-10 07:15:42 on dist-test-slave-gjrz"
server_key: "6ea23018467f366360c1f849728499ed"
server_key_iv: "36bee189278ea0e7e7867657d46d7e08"
server_key_version: "encryptionkey@0"
I20260810 07:15:42.825852 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "bf41c8a59fa048479959412135efcbce"
format_stamp: "Formatted at 2026-08-10 07:15:42 on dist-test-slave-gjrz"
server_key: "6ea23018467f366360c1f849728499ed"
server_key_iv: "36bee189278ea0e7e7867657d46d7e08"
server_key_version: "encryptionkey@0"
I20260810 07:15:42.829346 26675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:42.831585 26700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:42.832744 26675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:15:42.832856 26675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "bf41c8a59fa048479959412135efcbce"
format_stamp: "Formatted at 2026-08-10 07:15:42 on dist-test-slave-gjrz"
server_key: "6ea23018467f366360c1f849728499ed"
server_key_iv: "36bee189278ea0e7e7867657d46d7e08"
server_key_version: "encryptionkey@0"
I20260810 07:15:42.832947 26675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:42.849259 26675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:42.851841 26675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:42.852005 26675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:42.858611 26675 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:42513
I20260810 07:15:42.858626 26789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:42513 every 8 connection(s)
I20260810 07:15:42.859510 26675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:42.862134 26790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:42.866866 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 26675
I20260810 07:15:42.866993 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:42.867311 12758 external_mini_cluster.cc:1428] Setting key 44881a326c551c494aebd26358aeb3c7
I20260810 07:15:42.867538 26790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Bootstrap starting.
I20260810 07:15:42.869870 26790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:42.870793 26790 log.cc:824] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:42.872816 26790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: No bootstrap required, opened a new log
Aug 10 07:15:42 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:42.876717 26790 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:42.876986 26790 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:42.877054 26790 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41c8a59fa048479959412135efcbce, State: Initialized, Role: FOLLOWER
I20260810 07:15:42.877463 26790 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:42.877612 26790 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:42.877683 26790 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:42.877784 26790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:42.878638 26790 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:42.878943 26790 leader_election.cc:302] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf41c8a59fa048479959412135efcbce; no voters:
I20260810 07:15:42.879305 26790 leader_election.cc:288] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:42.879442 26797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:42.879671 26797 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 LEADER]: Becoming Leader. State: Replica: bf41c8a59fa048479959412135efcbce, State: Running, Role: LEADER
I20260810 07:15:42.879997 26797 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:42.880523 26790 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:42.881095 26797 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf41c8a59fa048479959412135efcbce. Latest consensus state: current_term: 1 leader_uuid: "bf41c8a59fa048479959412135efcbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } } }
I20260810 07:15:42.881225 26797 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:42.881712 26813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:42.882218 26800 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf41c8a59fa048479959412135efcbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } } }
I20260810 07:15:42.882326 26800 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:42.882359 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:42.868591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.12.117.190:42513)
0810 07:15:42.868996 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:42.869010 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:42.869044 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:15:42.869645 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:42.869662 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:42.870059 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:42.870408 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:42.871653 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:42.873062 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:42.873906 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:42.874606 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:42.874768 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:42.877247 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:42.877252 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:42.877266 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:42.877299 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:42.878904 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:42.879444 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:42.879451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:42.879457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:42.879605 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:42.879963 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:42.879967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:42.879969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:42.880442 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:42.880649 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:42.881036 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 07:15:42.881332 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":119,"threads_started":1}
I20260810 07:15:42.885090 26813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:42.891148 26813 catalog_manager.cc:1347] Generated new cluster ID: d7e995237e6441c8a9d415e32aef2c74
I20260810 07:15:42.891202 26813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:42.914323 26813 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:42.915642 26813 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:42.922044 26813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Generated new TSK 0
I20260810 07:15:42.922634 26813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:15:42.962469 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:43.087893 26843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:43.088120 26843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:43.088174 26843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:43.092103 26843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:43.092149 26843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:43.092202 26843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:43.097154 26843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:43.098007 26843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:43.098934 26843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:43.107836 26849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:43.107863 26853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:43.107895 26850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.108016 26843 server_base.cc:1034] running on GCE node
I20260810 07:15:43.108505 26843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:43.109058 26843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:43.110216 26843 hybrid_clock.cc:648] HybridClock initialized: now 1786346143110196 us; error 32 us; skew 500 ppm
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:43.112838 26843 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:15:43.113629 26843 webserver.cc:460] Webserver started at http://127.12.117.129:42381/ using document root <none> and password file <none>
I20260810 07:15:43.114138 26843 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:43.114199 26843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:43.114389 26843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:43.116397 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "4b2882e4db4b4f2f8d90a4759d820b8e"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "31d684c92f37ca12f0005eeb95007ab5"
server_key_iv: "20f459c6c186dc42a0e0db35cafbec8e"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.116940 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "4b2882e4db4b4f2f8d90a4759d820b8e"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "31d684c92f37ca12f0005eeb95007ab5"
server_key_iv: "20f459c6c186dc42a0e0db35cafbec8e"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.120472 26843 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:15:43.122686 26862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.123812 26843 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:43.123919 26843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "4b2882e4db4b4f2f8d90a4759d820b8e"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "31d684c92f37ca12f0005eeb95007ab5"
server_key_iv: "20f459c6c186dc42a0e0db35cafbec8e"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.124012 26843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:43.148880 26843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:43.151684 26843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:43.151858 26843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:43.152411 26843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:43.153204 26843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:43.153252 26843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.153303 26843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:43.153329 26843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.162036 26843 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:46391
I20260810 07:15:43.162070 27070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:46391 every 8 connection(s)
I20260810 07:15:43.162858 26843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:43.169499 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 26843
I20260810 07:15:43.169627 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:43.169924 12758 external_mini_cluster.cc:1428] Setting key 1bfcaee3051de038da2a74c1bf2a509f
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:43.176182 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.164581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:46731 (local address 127.12.117.190:42513)
0810 07:15:43.164742 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:43.164746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:43.165523 (+ 777us) server_negotiation.cc:407] Connection header received
0810 07:15:43.166728 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:43.166731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:43.166778 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:43.166848 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:43.167926 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.168454 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.169737 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.170025 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.170115 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.172529 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:43.172532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:43.172534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:43.172557 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:43.174070 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.174542 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.174546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.174548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.174645 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.174951 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.174954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.174956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.175161 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:43.175257 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:43.175846 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 07:15:43.176010 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 07:15:43.176831 27073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.164911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:46731)
0810 07:15:43.165372 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:43.165412 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:43.166530 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:43.166974 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:43.166980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:43.167339 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:43.167807 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.167821 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.168547 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.168549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.169565 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.169577 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.170178 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.170183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.170357 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.170878 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:43.170895 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:43.172368 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:43.174184 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.174189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.174200 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.174448 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.174726 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.174729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.174730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.174863 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.175366 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:43.175371 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:43.175598 (+ 227us) client_negotiation.cc:769] Sending connection context
0810 07:15:43.175758 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 07:15:43.176009 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":115,"threads_started":1}
I20260810 07:15:43.177986 27071 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42513
I20260810 07:15:43.178200 27071 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:43.178638 27071 heartbeater.cc:511] Master 127.12.117.190:42513 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:15:43.180137 26726 ts_manager.cc:194] Registered new tserver with Master: 4b2882e4db4b4f2f8d90a4759d820b8e (127.12.117.129:46391)
I20260810 07:15:43.181277 26726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:46731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:15:43.196050 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:43.304952 27076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:43.305171 27076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:43.305210 27076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:43.309095 27076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:43.309141 27076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:43.309195 27076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:43.314214 27076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:43.315078 27076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:43.315992 27076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:43.325196 27081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:43.325196 27085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.325285 27076 server_base.cc:1034] running on GCE node
W20260810 07:15:43.325223 27083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.325738 27076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:43.326300 27076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:43.327557 27076 hybrid_clock.cc:648] HybridClock initialized: now 1786346143327522 us; error 39 us; skew 500 ppm
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:43.330181 27076 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:15:43.331030 27076 webserver.cc:460] Webserver started at http://127.12.117.130:45695/ using document root <none> and password file <none>
I20260810 07:15:43.331629 27076 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:43.331691 27076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:43.331861 27076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:43.333771 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "31f10afda1e547d293517f1aa023c899"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "c38e9b760fc594a17d81bfad09322c90"
server_key_iv: "580c7c1a3a958da20f80423557b823f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.334282 27076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "31f10afda1e547d293517f1aa023c899"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "c38e9b760fc594a17d81bfad09322c90"
server_key_iv: "580c7c1a3a958da20f80423557b823f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.337867 27076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:43.340243 27097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.341419 27076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:43.341527 27076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "31f10afda1e547d293517f1aa023c899"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "c38e9b760fc594a17d81bfad09322c90"
server_key_iv: "580c7c1a3a958da20f80423557b823f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.341620 27076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:43.368821 27076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:43.371361 27076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:43.371532 27076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:43.372100 27076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:43.372838 27076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:43.372884 27076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.372920 27076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:43.372941 27076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.381126 27076 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:34143
I20260810 07:15:43.381191 27292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:34143 every 8 connection(s)
I20260810 07:15:43.382011 27076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:43.391775 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 27076
I20260810 07:15:43.391894 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:15:43.392194 12758 external_mini_cluster.cc:1428] Setting key e9a4b15c25efbe8b57ab9587231806ba
I20260810 07:15:43.394652 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.383670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:54401 (local address 127.12.117.190:42513)
0810 07:15:43.383784 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:43.383788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:43.384573 (+ 785us) server_negotiation.cc:407] Connection header received
0810 07:15:43.385780 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:43.385784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:43.385840 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:43.385912 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:43.387007 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.387590 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.388720 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.388944 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.389032 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.391294 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:43.391299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:43.391302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:43.391331 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:43.392721 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.393193 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.393196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.393198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.393290 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.393610 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.393614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.393616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.393803 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:43.393890 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:43.394367 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 07:15:43.394494 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:15:43.395388 27297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.384014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:54401)
0810 07:15:43.384440 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:43.384479 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:43.385583 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:43.386072 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:43.386079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:43.386419 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:43.386868 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.386881 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.387722 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.387725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.388597 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.388608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.389080 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.389085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.389250 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.389764 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:43.389780 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:43.391093 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:43.392827 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.392832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.392843 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.393101 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.393385 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.393388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.393389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.393519 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.393904 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:43.393911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:43.394140 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 07:15:43.394302 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 07:15:43.394599 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":107,"threads_started":1}
I20260810 07:15:43.396497 27294 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42513
I20260810 07:15:43.396709 27294 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:43.397161 27294 heartbeater.cc:511] Master 127.12.117.190:42513 requested a full tablet report, sending...
I20260810 07:15:43.398145 26726 ts_manager.cc:194] Registered new tserver with Master: 31f10afda1e547d293517f1aa023c899 (127.12.117.130:34143)
I20260810 07:15:43.398691 26726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:54401
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:15:43.418143 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:43.527030 27302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:43.527256 27302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:43.527326 27302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:43.531236 27302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:43.531319 27302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:43.531388 27302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:43.536448 27302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:43.537345 27302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:43.538281 27302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:43.547245 27311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.547334 27302 server_base.cc:1034] running on GCE node
W20260810 07:15:43.547257 27314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:43.547243 27312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.547863 27302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:43.548431 27302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:43.549607 27302 hybrid_clock.cc:648] HybridClock initialized: now 1786346143549587 us; error 27 us; skew 500 ppm
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:43.552234 27302 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:15:43.553049 27302 webserver.cc:460] Webserver started at http://127.12.117.131:35313/ using document root <none> and password file <none>
I20260810 07:15:43.553561 27302 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:43.553625 27302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:43.553817 27302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:43.555764 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "d7162643e54a4159bd6a8fb634eb3cec"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "a92501aa33a909de378dea849d8eb31a"
server_key_iv: "d2ff0b1002391dac1cf9d38a28af252a"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.556277 27302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "d7162643e54a4159bd6a8fb634eb3cec"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "a92501aa33a909de378dea849d8eb31a"
server_key_iv: "d2ff0b1002391dac1cf9d38a28af252a"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.559811 27302 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:15:43.562040 27329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.563179 27302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:43.563305 27302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "d7162643e54a4159bd6a8fb634eb3cec"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "a92501aa33a909de378dea849d8eb31a"
server_key_iv: "d2ff0b1002391dac1cf9d38a28af252a"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.563408 27302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:43.590597 27302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:43.593495 27302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:43.593652 27302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:43.594162 27302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:43.594921 27302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:43.594969 27302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.595019 27302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:43.595049 27302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.603091 27302 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:37713
I20260810 07:15:43.603123 27513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:37713 every 8 connection(s)
I20260810 07:15:43.603935 27302 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:15:43.613644 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 27302
I20260810 07:15:43.613773 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:15:43.614045 12758 external_mini_cluster.cc:1428] Setting key 830f2b80198323f41da7c0aeb7a49930
I20260810 07:15:43.616799 26793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.605592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:56505 (local address 127.12.117.190:42513)
0810 07:15:43.605729 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:43.605733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:43.606473 (+ 740us) server_negotiation.cc:407] Connection header received
0810 07:15:43.607736 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:43.607740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:43.607799 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:43.607872 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:43.608986 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.609522 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.610772 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.610998 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.611089 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.613303 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:43.613306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:43.613309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:43.613334 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:43.614777 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.615390 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.615393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.615395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.615494 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:43.615827 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:43.615831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:43.615832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:43.616057 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:43.616153 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:43.616581 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 07:15:43.616687 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:15:43.617518 27518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:43.605917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:56505)
0810 07:15:43.606335 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:43.606378 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:43.607537 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:43.608079 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:43.608086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:43.608402 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:43.608833 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.608846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.609643 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.609646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.610631 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:43.610644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:43.611138 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:43.611141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:43.611331 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:43.611775 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:43.611791 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:43.613153 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:43.614904 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.614910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.614923 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.615296 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.615584 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:15:43.615586 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:15:43.615587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:15:43.615718 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:15:43.616165 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:43.616171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:43.616385 (+ 214us) client_negotiation.cc:769] Sending connection context
0810 07:15:43.616531 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 07:15:43.616756 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":110,"threads_started":1}
I20260810 07:15:43.618474 27514 heartbeater.cc:347] Connected to a master server at 127.12.117.190:42513
I20260810 07:15:43.618674 27514 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:43.619092 27514 heartbeater.cc:511] Master 127.12.117.190:42513 requested a full tablet report, sending...
I20260810 07:15:43.620012 26726 ts_manager.cc:194] Registered new tserver with Master: d7162643e54a4159bd6a8fb634eb3cec (127.12.117.131:37713)
I20260810 07:15:43.620545 26726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:56505
I20260810 07:15:43.627553 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
Entry for principal oryx/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab.
WARNING: no policy specified for oryx/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
Entry for principal oryx/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab.
WARNING: no policy specified for oryx/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
Entry for principal oryx/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab.
I20260810 07:15:43.692126 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 26843
I20260810 07:15:43.711613 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 27076
I20260810 07:15:43.726863 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 27302
I20260810 07:15:43.734071 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 26675
I20260810 07:15:43.745301 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42513
--webserver_interface=127.12.117.190
--webserver_port=42661
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:42513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:15:43.852238 27532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:43.852459 27532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:43.852494 27532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:43.856287 27532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:43.856328 27532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:43.856344 27532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:43.856359 27532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:43.856372 27532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:43.861721 27532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:42513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:42513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=42661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:43.862648 27532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:43.863535 27532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:43.871244 27544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:43.871249 27540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.871359 27532 server_base.cc:1034] running on GCE node
W20260810 07:15:43.871253 27547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:43.872030 27532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:43.873019 27532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:43.874192 27532 hybrid_clock.cc:648] HybridClock initialized: now 1786346143874177 us; error 28 us; skew 500 ppm
Aug 10 07:15:43 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346143, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:43.876842 27532 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:15:43.877661 27532 webserver.cc:460] Webserver started at http://127.12.117.190:42661/ using document root <none> and password file <none>
I20260810 07:15:43.878180 27532 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:43.878262 27532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:43.881786 27532 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:43.883816 27565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:43.885138 27532 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:43.885262 27532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "bf41c8a59fa048479959412135efcbce"
format_stamp: "Formatted at 2026-08-10 07:15:42 on dist-test-slave-gjrz"
server_key: "6ea23018467f366360c1f849728499ed"
server_key_iv: "36bee189278ea0e7e7867657d46d7e08"
server_key_version: "encryptionkey@0"
I20260810 07:15:43.885772 27532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:43.902314 27532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:43.904814 27532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:43.904965 27532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:43.911283 27532 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:42513
I20260810 07:15:43.911348 27667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:42513 every 8 connection(s)
I20260810 07:15:43.912235 27532 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:43.916244 27669 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:15:43.917973 27669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Bootstrap starting.
I20260810 07:15:43.921343 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 27532
I20260810 07:15:43.921758 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:46391
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:42513
--webserver_port=42381
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:15:43.933738 27669 log.cc:824] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:43.939831 27669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:15:43.940141 27669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Bootstrap complete.
I20260810 07:15:43.944103 27669 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:43.944484 27669 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf41c8a59fa048479959412135efcbce, State: Initialized, Role: FOLLOWER
I20260810 07:15:43.944980 27669 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:43.945096 27669 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:43.945158 27669 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:43.945248 27669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:15:43.947062 27669 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:43.947387 27669 leader_election.cc:302] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf41c8a59fa048479959412135efcbce; no voters:
I20260810 07:15:43.947662 27669 leader_election.cc:288] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:15:43.947754 27674 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:15:43.947922 27674 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [term 2 LEADER]: Becoming Leader. State: Replica: bf41c8a59fa048479959412135efcbce, State: Running, Role: LEADER
I20260810 07:15:43.948200 27674 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } }
I20260810 07:15:43.948803 27669 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:43.949086 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf41c8a59fa048479959412135efcbce. Latest consensus state: current_term: 2 leader_uuid: "bf41c8a59fa048479959412135efcbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } } }
I20260810 07:15:43.949200 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:43.949558 27681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:43.949568 27675 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf41c8a59fa048479959412135efcbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf41c8a59fa048479959412135efcbce" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 42513 } } }
I20260810 07:15:43.949795 27675 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:43.952661 27681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:43.953372 27681 catalog_manager.cc:1259] Loaded cluster ID: d7e995237e6441c8a9d415e32aef2c74
I20260810 07:15:43.953425 27681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:43.955742 27681 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:43.956387 27681 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf41c8a59fa048479959412135efcbce: Loaded TSK: 0
I20260810 07:15:43.957552 27681 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:15:44.030468 27671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:44.030686 27671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:44.030742 27671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:44.034627 27671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:44.034679 27671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:44.034756 27671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:44.039745 27671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.129.keytab
--principal=oryx/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:46391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=42381
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:44.040618 27671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:44.041527 27671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:44.051394 27709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.051406 27704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.051394 27706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:44.052211 27671 server_base.cc:1034] running on GCE node
I20260810 07:15:44.052584 27671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:44.053165 27671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:44.054333 27671 hybrid_clock.cc:648] HybridClock initialized: now 1786346144054314 us; error 34 us; skew 500 ppm
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346144, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:44.056898 27671 init.cc:377] Logged in from keytab as oryx/127.12.117.129@KRBTEST.COM (short username oryx)
I20260810 07:15:44.057857 27671 webserver.cc:460] Webserver started at http://127.12.117.129:42381/ using document root <none> and password file <none>
I20260810 07:15:44.058373 27671 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:44.058444 27671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:44.062068 27671 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.064041 27720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.065237 27671 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.065335 27671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "4b2882e4db4b4f2f8d90a4759d820b8e"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "31d684c92f37ca12f0005eeb95007ab5"
server_key_iv: "20f459c6c186dc42a0e0db35cafbec8e"
server_key_version: "encryptionkey@0"
I20260810 07:15:44.065836 27671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:44.079692 27671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:44.082492 27671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:44.082661 27671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:44.083171 27671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:44.083959 27671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:44.084013 27671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.084067 27671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:44.084095 27671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.092594 27671 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:46391
I20260810 07:15:44.092624 27928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:46391 every 8 connection(s)
I20260810 07:15:44.093502 27671 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:44.098148 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 27671
I20260810 07:15:44.098642 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:34143
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:42513
--webserver_port=45695
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.105666 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.095429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:59775)
0810 07:15:44.096012 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.096050 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.096755 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.097690 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.097696 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.098095 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.098707 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.098729 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.100038 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.100042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.100943 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.100954 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.101852 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.101856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.102059 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.102563 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.102582 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.105034 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":122,"threads_started":1}
W20260810 07:15:44.106179 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.095314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59775 (local address 127.12.117.190:42513)
0810 07:15:44.095742 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.095763 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.096155 (+ 392us) server_negotiation.cc:407] Connection header received
0810 07:15:44.097070 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.097081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.097353 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.097615 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.098901 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.099899 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.101074 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.101692 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.101813 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.105309 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:59775: BlockingRecv error: recv got EOF from 127.12.117.129:59775 (error 108)
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":90,"threads_started":1}
W20260810 07:15:44.106316 27934 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:42513 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.117.190:42513: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.111680 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.106714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:59715)
0810 07:15:44.106839 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.106864 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.106954 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.107179 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.107182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.107362 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.107487 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.107496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.108154 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.108157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.109004 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.109016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.109417 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.109420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.109518 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.109899 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.109922 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.111576 (+ 1654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 07:15:44.112022 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.106783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59715 (local address 127.12.117.190:42513)
0810 07:15:44.106882 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.106885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.106901 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:15:44.107040 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.107042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.107079 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.107141 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.107588 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.108053 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.109115 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.109316 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.109389 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.111712 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:59715: BlockingRecv error: recv got EOF from 127.12.117.129:59715 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.116410 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.112129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:60783)
0810 07:15:44.112262 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.112275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.112343 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.112553 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.112556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.112681 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.112743 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.112748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.113400 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.113403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.113893 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.113900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.114290 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.114293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.114366 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.114723 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.114734 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.116304 (+ 1570us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260810 07:15:44.116529 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.112207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:60783 (local address 127.12.117.190:42513)
0810 07:15:44.112307 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.112311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.112324 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:44.112421 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.112424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.112461 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.112527 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.112841 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.113311 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.113994 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.114198 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.114279 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.116414 (+ 2135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:60783: BlockingRecv error: recv got EOF from 127.12.117.129:60783 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 07:15:44.116614 27934 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:15:44.208819 27940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:44.209043 27940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:44.209100 27940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:44.212939 27940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:44.212989 27940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:44.213064 27940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:15:44.217981 27940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.130.keytab
--principal=oryx/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:34143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=45695
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:44.218843 27940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:44.219753 27940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:44.228324 27953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.228339 27954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.228329 27956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:44.228379 27940 server_base.cc:1034] running on GCE node
I20260810 07:15:44.228869 27940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:44.229406 27940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:44.230556 27940 hybrid_clock.cc:648] HybridClock initialized: now 1786346144230537 us; error 28 us; skew 500 ppm
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346144, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:44.233045 27940 init.cc:377] Logged in from keytab as oryx/127.12.117.130@KRBTEST.COM (short username oryx)
I20260810 07:15:44.233821 27940 webserver.cc:460] Webserver started at http://127.12.117.130:45695/ using document root <none> and password file <none>
I20260810 07:15:44.234310 27940 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:44.234391 27940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:44.237953 27940 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.239888 27967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.241004 27940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.241115 27940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "31f10afda1e547d293517f1aa023c899"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "c38e9b760fc594a17d81bfad09322c90"
server_key_iv: "580c7c1a3a958da20f80423557b823f5"
server_key_version: "encryptionkey@0"
I20260810 07:15:44.241595 27940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:44.253681 27940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:44.256273 27940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:44.256438 27940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:44.256975 27940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:44.257769 27940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:44.257827 27940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.257879 27940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:44.257907 27940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.266247 27940 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:34143
I20260810 07:15:44.266275 28165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:34143 every 8 connection(s)
I20260810 07:15:44.267141 27940 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:15:44.275517 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 27940
I20260810 07:15:44.275923 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:37713
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:42513
--webserver_port=35313
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:45089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:44.278717 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.269020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:41399 (local address 127.12.117.190:42513)
0810 07:15:44.269175 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.269180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.269993 (+ 813us) server_negotiation.cc:407] Connection header received
0810 07:15:44.271465 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.271469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.271529 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.271617 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.272989 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.273769 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.274887 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.275115 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.275204 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.278619 (+ 3415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:41399: BlockingRecv error: recv got EOF from 127.12.117.130:41399 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:15:44.279419 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.269404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:41399)
0810 07:15:44.269834 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.269875 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.271196 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.271804 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.271812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.272281 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.272844 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.272858 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.273902 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.273905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.274765 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.274774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.275247 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.275252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.275450 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.275955 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.275971 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.278501 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
W20260810 07:15:44.279865 28166 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:42513 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.117.190:42513: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.285138 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.280239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:45563)
0810 07:15:44.280381 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.280394 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.280462 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.280696 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.280699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.280830 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.280918 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.280923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.281851 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.281853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.282380 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.282386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.282893 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.282895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.282958 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.283415 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.283426 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.285022 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260810 07:15:44.285231 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.280349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:45563 (local address 127.12.117.190:42513)
0810 07:15:44.280490 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.280494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.280508 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:44.280546 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.280549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.280589 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.280670 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.281023 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.281759 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.282483 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.282797 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.282905 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.285136 (+ 2231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:45563: BlockingRecv error: recv got EOF from 127.12.117.130:45563 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.290232 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.285732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:44359)
0810 07:15:44.285940 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.285952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.286027 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.286223 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.286225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.286359 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.286420 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.286424 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.287079 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.287082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.287617 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.287626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.288012 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.288015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.288087 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.288495 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.288504 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.290126 (+ 1622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154}
W20260810 07:15:44.290400 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.285838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:44359 (local address 127.12.117.190:42513)
0810 07:15:44.285939 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.285943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.285993 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:15:44.286099 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.286102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.286139 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.286198 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.286519 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.286982 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.287720 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.287920 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.288002 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.290228 (+ 2226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:44359: BlockingRecv error: recv got EOF from 127.12.117.130:44359 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 07:15:44.290567 28166 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:15:44.384591 28171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:44.384889 28171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:44.384954 28171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:44.388898 28171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:44.388950 28171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:44.389003 28171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:15:44.393826 28171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:45089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/krb5kdc/oryx_127.12.117.131.keytab
--principal=oryx/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:37713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=35313
--tserver_master_addrs=127.12.117.190:42513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:44.394635 28171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:44.395539 28171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:44.404342 28182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.404364 28190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:44.404350 28184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:44.404526 28171 server_base.cc:1034] running on GCE node
I20260810 07:15:44.405014 28171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:44.405597 28171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:44.406790 28171 hybrid_clock.cc:648] HybridClock initialized: now 1786346144406770 us; error 32 us; skew 500 ppm
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346144, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:15:44.409371 28171 init.cc:377] Logged in from keytab as oryx/127.12.117.131@KRBTEST.COM (short username oryx)
I20260810 07:15:44.410176 28171 webserver.cc:460] Webserver started at http://127.12.117.131:35313/ using document root <none> and password file <none>
I20260810 07:15:44.410676 28171 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:44.410756 28171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:44.414269 28171 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.416189 28201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.417258 28171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:15:44.417362 28171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "d7162643e54a4159bd6a8fb634eb3cec"
format_stamp: "Formatted at 2026-08-10 07:15:43 on dist-test-slave-gjrz"
server_key: "a92501aa33a909de378dea849d8eb31a"
server_key_iv: "d2ff0b1002391dac1cf9d38a28af252a"
server_key_version: "encryptionkey@0"
I20260810 07:15:44.417845 28171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:44.432932 28171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:44.435827 28171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:44.435986 28171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:44.436487 28171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:44.437207 28171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:44.437255 28171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.437304 28171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:44.437331 28171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:44.445601 28171 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:37713
I20260810 07:15:44.445634 28406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:37713 every 8 connection(s)
I20260810 07:15:44.446463 28171 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:15:44.452697 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 28171
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 07:15:44.457338 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.448251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:56955 (local address 127.12.117.190:42513)
0810 07:15:44.448391 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.448396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.449195 (+ 799us) server_negotiation.cc:407] Connection header received
0810 07:15:44.450480 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.450484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.450540 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.450612 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.451800 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.452316 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.453668 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.453974 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.454067 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.457215 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:56955: BlockingRecv error: recv got EOF from 127.12.117.131:56955 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:15:44.457858 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.448620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:56955)
0810 07:15:44.449059 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.449098 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.450263 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.450788 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.450795 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.451120 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.451658 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.451675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.452437 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.452441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.453552 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.453563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.454139 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.454143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.454361 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.454862 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.454877 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.457092 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":96,"threads_started":1}
W20260810 07:15:44.458392 28410 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:42513 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.117.190:42513: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.462592 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.453575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53070 (local address 127.12.117.190:42513)
0810 07:15:44.453784 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.453788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.453800 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:44.453829 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.453832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.453873 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.453969 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.454808 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.455341 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.456655 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.456878 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.456966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.458780 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:44.458787 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:44.458804 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:15:44.458834 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:44.460420 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:44.460889 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:44.460895 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:44.460901 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:44.461050 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:15:44.461317 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:15:44.461320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:15:44.461322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:15:44.461724 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:44.461890 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:44.462213 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 07:15:44.462435 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.463827 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.458773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:56309)
0810 07:15:44.458917 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.458930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.458992 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.459189 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.459192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.459361 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.459454 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.459459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.460141 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.460144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.460785 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.460796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.461301 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.461306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.461381 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.461766 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.461775 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.463721 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260810 07:15:44.463899 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.458853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:56309 (local address 127.12.117.190:42513)
0810 07:15:44.458994 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.458998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.459012 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:44.459048 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.459050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.459090 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.459165 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.459561 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.460040 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.460889 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.461201 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.461271 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.463823 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:56309: BlockingRecv error: recv got EOF from 127.12.117.131:56309 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:44 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:44.468744 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:44.464416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:49303)
0810 07:15:44.464751 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:44.464762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:44.464842 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:44.465055 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:44.465058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:44.465187 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:44.465250 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.465255 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.465904 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.465907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.466393 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:44.466400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.466805 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.466808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:44.466882 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.467191 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:44.467201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:44.468670 (+ 1469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282}
W20260810 07:15:44.468863 27936 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:44.464488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:49303 (local address 127.12.117.190:42513)
0810 07:15:44.464602 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:44.464615 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:44.464824 (+ 209us) server_negotiation.cc:407] Connection header received
0810 07:15:44.464914 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:44.464916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:44.464956 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:44.465020 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:44.465354 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.465809 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.466504 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:44.466714 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:44.466795 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:44.468771 (+ 1976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:49303: BlockingRecv error: recv got EOF from 127.12.117.131:49303 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:15:44.468915 28410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:45.122225 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:45.117134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:54885)
0810 07:15:45.117409 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:45.117425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:45.117521 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:45.117760 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:45.117762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:45.117930 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:45.118061 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.118066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.118808 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.118810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.119403 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.119410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.119789 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.119792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.119859 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.120343 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:45.120358 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:45.122119 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
W20260810 07:15:45.122380 28421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:45.117193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:54885 (local address 127.12.117.190:42513)
0810 07:15:45.117474 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:45.117479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:45.117494 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:15:45.117614 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:45.117616 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:45.117659 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:45.117731 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:45.118185 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.118728 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.119488 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.119709 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.119781 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.122254 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:54885: BlockingRecv error: recv got EOF from 127.12.117.129:54885 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":74,"threads_started":1}
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:45.295987 28425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:45.291080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:33695)
0810 07:15:45.291350 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:45.291362 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:45.291455 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:45.291662 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:45.291664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:45.291829 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:45.291923 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.291928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.292655 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.292657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.293225 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.293232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.293683 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.293685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.293748 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.294186 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:45.294197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:45.295869 (+ 1672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":66,"threads_started":1}
W20260810 07:15:45.296100 28421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:45.291149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:33695 (local address 127.12.117.190:42513)
0810 07:15:45.291296 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:45.291300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:45.291419 (+ 119us) server_negotiation.cc:407] Connection header received
0810 07:15:45.291535 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:45.291537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:45.291580 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:45.291637 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:45.292045 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.292575 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.293346 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.293579 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.293665 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.296000 (+ 2335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:33695: BlockingRecv error: recv got EOF from 127.12.117.130:33695 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:45 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:45.474308 28426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:45.469445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:50953)
0810 07:15:45.469700 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:45.469714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:45.469814 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:45.470020 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:45.470022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:45.470203 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:45.470299 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.470304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.470997 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.471000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.471630 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:45.471640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.472015 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.472018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:45.472084 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.472497 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:45.472507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:45.474186 (+ 1679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
W20260810 07:15:45.474431 28421 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:45.469546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:50953 (local address 127.12.117.190:42513)
0810 07:15:45.469659 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:45.469663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:45.469779 (+ 116us) server_negotiation.cc:407] Connection header received
0810 07:15:45.469886 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:45.469887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:45.469931 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:45.469994 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:45.470391 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.470908 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.471718 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:45.471931 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:45.472003 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:45.474314 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:50953: BlockingRecv error: recv got EOF from 127.12.117.131:50953 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:46.128154 28430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:46.122879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:38399)
0810 07:15:46.123163 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:46.123177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:46.123282 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:46.123510 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:46.123512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:46.123696 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:46.123797 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.123802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.124599 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.124601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.125150 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.125156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.125648 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.125652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.125743 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.126248 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:46.126261 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:46.128034 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
W20260810 07:15:46.128270 28431 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:46.122941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:38399 (local address 127.12.117.190:42513)
0810 07:15:46.123205 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:46.123209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:46.123222 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:46.123362 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:46.123365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:46.123408 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:46.123486 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:46.123880 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.124519 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.125234 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.125522 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.125622 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.128163 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:38399: BlockingRecv error: recv got EOF from 127.12.117.129:38399 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":103,"threads_started":1}
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:46.301759 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:46.296635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:38709)
0810 07:15:46.296890 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:46.296907 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:46.297025 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:46.297271 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:46.297274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:46.297481 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:46.297579 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.297584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.298321 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.298325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.298892 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.298900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.299363 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.299366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.299460 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.299885 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:46.299896 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:46.301587 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
W20260810 07:15:46.301826 28431 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:46.296732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:38709 (local address 127.12.117.190:42513)
0810 07:15:46.296891 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:46.296896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:46.296993 (+ 97us) server_negotiation.cc:407] Connection header received
0810 07:15:46.297126 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:46.297129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:46.297177 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:46.297241 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:46.297687 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.298219 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.298999 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.299226 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.299358 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.301726 (+ 2368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:38709: BlockingRecv error: recv got EOF from 127.12.117.130:38709 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:46 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:46.484586 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:46.474976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:49611)
0810 07:15:46.475241 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:46.475256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:46.475383 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:46.475618 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:46.475621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:46.475813 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:46.475909 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.475915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.476674 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.476678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.477250 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:46.477258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.477693 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.477697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:46.477784 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.482445 (+ 4661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:46.482457 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:46.484411 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
W20260810 07:15:46.484668 28431 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:46.475110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:49611 (local address 127.12.117.190:42513)
0810 07:15:46.475241 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:46.475246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:46.475339 (+ 93us) server_negotiation.cc:407] Connection header received
0810 07:15:46.475467 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:46.475470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:46.475518 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:46.475583 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:46.476025 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.476563 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.477365 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:46.477585 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:46.477678 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:46.484550 (+ 6872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:49611: BlockingRecv error: recv got EOF from 127.12.117.131:49611 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:47.133898 28434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:47.128821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:57023)
0810 07:15:47.129122 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:47.129135 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:47.129232 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:47.129509 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:47.129511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:47.129686 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:47.129786 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.129792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.130498 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.130500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.131063 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.131070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.131514 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.131517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.131576 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.132017 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:47.132027 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:47.133773 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":116,"threads_started":1}
W20260810 07:15:47.134016 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:47.128901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:57023 (local address 127.12.117.190:42513)
0810 07:15:47.129115 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:47.129119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:47.129199 (+ 80us) server_negotiation.cc:407] Connection header received
0810 07:15:47.129350 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:47.129353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:47.129401 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:47.129486 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:47.129882 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.130403 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.131157 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.131414 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.131510 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.133904 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:57023: BlockingRecv error: recv got EOF from 127.12.117.129:57023 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":59,"threads_started":1}
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:47.307869 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:47.302467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:39453)
0810 07:15:47.302714 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:47.302727 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:47.302826 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:47.303027 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:47.303029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:47.303198 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:47.303332 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.303340 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.304191 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.304195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.304896 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.304907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.305317 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.305319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.305381 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.305857 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:47.305869 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:47.307672 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
W20260810 07:15:47.307896 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:47.302535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:39453 (local address 127.12.117.190:42513)
0810 07:15:47.302689 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:47.302693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:47.302788 (+ 95us) server_negotiation.cc:407] Connection header received
0810 07:15:47.302895 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:47.302898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:47.302940 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:47.303005 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:47.303451 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.304093 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.305004 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.305229 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.305310 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.307799 (+ 2489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:39453: BlockingRecv error: recv got EOF from 127.12.117.130:39453 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:47 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:47.490325 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:47.485395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:42301)
0810 07:15:47.485673 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:47.485688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:47.485786 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:47.485989 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:47.485991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:47.486160 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:47.486258 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.486264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.486917 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.486919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.487498 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:47.487506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.487886 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.487889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:47.487961 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.488443 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:47.488458 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:47.490191 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":86,"threads_started":1}
W20260810 07:15:47.490414 28435 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:47.485480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:42301 (local address 127.12.117.190:42513)
0810 07:15:47.485605 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:47.485608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:47.485748 (+ 140us) server_negotiation.cc:407] Connection header received
0810 07:15:47.485853 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:47.485855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:47.485899 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:47.485964 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:47.486340 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.486842 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.487586 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:47.487795 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:47.487868 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:47.490317 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:42301: BlockingRecv error: recv got EOF from 127.12.117.131:42301 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:48.139786 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:48.134564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:39195)
0810 07:15:48.134812 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:48.134826 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:48.134913 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:48.135151 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:48.135154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:48.135362 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:48.135470 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.135477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.136252 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.136254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.136827 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.136834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.137299 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.137301 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.137360 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.137806 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:48.137817 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:48.139630 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
W20260810 07:15:48.139894 28442 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:48.134630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:39195 (local address 127.12.117.190:42513)
0810 07:15:48.134865 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:48.134868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:48.134882 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:48.134999 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:48.135002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:48.135045 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:48.135118 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:48.135609 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.136168 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.136964 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.137199 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.137285 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.139778 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:39195: BlockingRecv error: recv got EOF from 127.12.117.129:39195 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:48.313776 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:48.308569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:35013)
0810 07:15:48.308853 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:48.308866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:48.308974 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:48.309222 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:48.309225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:48.309420 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:48.309525 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.309531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.310342 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.310346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.310902 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.310911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.311372 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.311375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.311465 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.311897 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:48.311907 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:48.313662 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":99,"threads_started":1}
W20260810 07:15:48.313918 28442 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:48.308658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:35013 (local address 127.12.117.190:42513)
0810 07:15:48.308798 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:48.308802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:48.308937 (+ 135us) server_negotiation.cc:407] Connection header received
0810 07:15:48.309063 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:48.309066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:48.309117 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:48.309186 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:48.309652 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.310233 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.311021 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.311248 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.311368 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.313799 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:35013: BlockingRecv error: recv got EOF from 127.12.117.130:35013 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:48 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:48.500420 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:48.491051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:35397)
0810 07:15:48.491797 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:48.491810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:48.491925 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:48.492242 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:48.492245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:48.492461 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:48.492588 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.492597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.493693 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.493698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.494570 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:48.494581 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.495318 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.495323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:48.495420 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.495943 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:48.495958 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:48.500266 (+ 4308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":94,"threads_started":1}
W20260810 07:15:48.500514 28442 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:48.491175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35397 (local address 127.12.117.190:42513)
0810 07:15:48.491389 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:48.491394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:48.491895 (+ 501us) server_negotiation.cc:407] Connection header received
0810 07:15:48.492052 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:48.492055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:48.492115 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:48.492197 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:48.492765 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.493571 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.494763 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:48.495164 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:48.495289 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:48.500394 (+ 5105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:35397: BlockingRecv error: recv got EOF from 127.12.117.131:35397 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:49.145845 28447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:49.140675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:45995)
0810 07:15:49.140924 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:49.140937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:49.141030 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:49.141248 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:49.141250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:49.141434 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:49.141528 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.141534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.142236 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.142238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.142812 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.142819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.143231 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.143234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.143331 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.143877 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:49.143889 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:49.145726 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
W20260810 07:15:49.145963 28448 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:49.140724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45995 (local address 127.12.117.190:42513)
0810 07:15:49.140956 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:49.140960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:49.140991 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:15:49.141111 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:49.141113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:49.141156 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:49.141237 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:49.141615 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.142155 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.142909 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.143144 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.143221 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.145847 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:45995: BlockingRecv error: recv got EOF from 127.12.117.129:45995 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:49.320016 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:49.314453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:60665)
0810 07:15:49.314727 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:49.314743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:49.314851 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:49.315094 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:49.315097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:49.315343 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:49.315444 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.315451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.316229 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.316233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.316811 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.316819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.317249 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.317253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.317337 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.317797 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:49.317809 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:49.319831 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
W20260810 07:15:49.320075 28448 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:49.314639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:60665 (local address 127.12.117.190:42513)
0810 07:15:49.314763 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:49.314768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:49.314800 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:15:49.314940 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:49.314943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:49.314990 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:49.315055 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:49.315557 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.316120 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.316916 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.317147 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.317236 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.319975 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:60665: BlockingRecv error: recv got EOF from 127.12.117.130:60665 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:49 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:49.510577 28450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:49.501324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:44781)
0810 07:15:49.501703 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:49.501719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:49.501819 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:49.502949 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:49.502953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:49.503188 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:49.503352 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.503361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.504382 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.504386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.505417 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:49.505432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.507712 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.507715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:49.507804 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.508452 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:49.508468 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:49.510463 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":84,"threads_started":1}
W20260810 07:15:49.510756 28448 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:49.502574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:44781 (local address 127.12.117.190:42513)
0810 07:15:49.502700 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:49.502704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:49.502717 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:49.502758 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:49.502761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:49.502810 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:49.502885 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:49.503480 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.504256 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.507208 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:49.507591 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:49.507700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:49.510638 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:44781: BlockingRecv error: recv got EOF from 127.12.117.131:44781 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:50.151741 28451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:50.146545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:34091)
0810 07:15:50.146833 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:50.146847 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:50.146945 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:50.147159 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:50.147161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:50.147355 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:50.147450 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.147455 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.148135 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.148137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.148685 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.148692 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.149048 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.149051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.149152 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.149596 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:50.149607 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:50.151580 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":100,"threads_started":1}
W20260810 07:15:50.151863 28454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:50.146615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:34091 (local address 127.12.117.190:42513)
0810 07:15:50.146838 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:50.146841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:50.146906 (+ 65us) server_negotiation.cc:407] Connection header received
0810 07:15:50.147014 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:50.147016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:50.147059 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:50.147142 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:50.147531 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.148054 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.148763 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.148970 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.149037 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.151726 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:34091: BlockingRecv error: recv got EOF from 127.12.117.129:34091 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":60,"threads_started":1}
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:50.326036 28455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:50.320878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:58787)
0810 07:15:50.321223 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:50.321236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:50.321369 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:50.321637 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:50.321640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:50.321841 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:50.321948 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.321954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.322698 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.322702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.323289 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.323298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.323728 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.323731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.323814 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.324242 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:50.324253 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:50.325921 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":154,"threads_started":1}
W20260810 07:15:50.326186 28454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:50.320930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:58787 (local address 127.12.117.190:42513)
0810 07:15:50.321081 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:50.321086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:50.321332 (+ 246us) server_negotiation.cc:407] Connection header received
0810 07:15:50.321464 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:50.321467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:50.321539 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:50.321613 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:50.322063 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.322595 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.323401 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.323630 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.323716 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.326076 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:58787: BlockingRecv error: recv got EOF from 127.12.117.130:58787 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:50 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:50.516044 28456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:50.511227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:43107)
0810 07:15:50.511506 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:50.511520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:50.511625 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:50.511827 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:50.511829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:50.512009 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:50.512108 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.512113 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.512768 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.512770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.513314 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:50.513321 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.513669 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.513671 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:50.513732 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.514170 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:50.514180 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:50.515895 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":91,"threads_started":1}
W20260810 07:15:50.516144 28454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:50.511306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:43107 (local address 127.12.117.190:42513)
0810 07:15:50.511418 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:50.511422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:50.511578 (+ 156us) server_negotiation.cc:407] Connection header received
0810 07:15:50.511694 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:50.511696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:50.511740 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:50.511802 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:50.512190 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.512693 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.513392 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:50.513594 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:50.513657 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:50.516043 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:43107: BlockingRecv error: recv got EOF from 127.12.117.131:43107 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:51.157759 28457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:51.152379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:42719)
0810 07:15:51.152695 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:51.152711 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:51.152807 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:51.153076 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:51.153079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:51.153260 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:51.153358 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.153364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.154106 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.154109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.154682 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.154691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.155063 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.155065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.155130 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.155632 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:51.155644 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:51.157643 (+ 1999us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":69,"threads_started":1}
W20260810 07:15:51.157884 28458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:51.152498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:42719 (local address 127.12.117.190:42513)
0810 07:15:51.152747 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:51.152750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:51.152764 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:51.152883 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:51.152885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:51.152928 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:51.153000 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:51.153461 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.153990 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.154770 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.154979 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.155045 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.157769 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:42719: BlockingRecv error: recv got EOF from 127.12.117.129:42719 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":78,"threads_started":1}
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:51.331861 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:51.326689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:50357)
0810 07:15:51.326974 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:51.326986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:51.327098 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:51.327355 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:51.327358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:51.327551 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:51.327656 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.327662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.328422 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.328426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.328994 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.329003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.329451 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.329454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.329538 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.329976 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:51.329987 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:51.331724 (+ 1737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":69,"threads_started":1}
W20260810 07:15:51.331974 28458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:51.326755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:50357 (local address 127.12.117.190:42513)
0810 07:15:51.326896 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:51.326902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:51.327055 (+ 153us) server_negotiation.cc:407] Connection header received
0810 07:15:51.327185 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:51.327188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:51.327240 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:51.327331 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:51.327773 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.328315 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.329116 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.329346 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.329437 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.331860 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:50357: BlockingRecv error: recv got EOF from 127.12.117.130:50357 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:51 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:51.525081 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:51.516764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:52549)
0810 07:15:51.517063 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:51.517078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:51.517176 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:51.517372 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:51.517374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:51.517579 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:51.517677 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.517683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.518345 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.518348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.518903 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:51.518910 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.519254 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.519257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:51.519352 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.523319 (+ 3967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:51.523331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:51.524939 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":83,"threads_started":1}
W20260810 07:15:51.525171 28458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:51.516814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:52549 (local address 127.12.117.190:42513)
0810 07:15:51.516936 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:51.516939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:51.517134 (+ 195us) server_negotiation.cc:407] Connection header received
0810 07:15:51.517238 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:51.517240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:51.517285 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:51.517348 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:51.517763 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.518268 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.518979 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:51.519182 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:51.519246 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:51.525066 (+ 5820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:52549: BlockingRecv error: recv got EOF from 127.12.117.131:52549 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:52.163764 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:52.158445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:47725)
0810 07:15:52.158690 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:52.158704 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:52.158791 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:52.159012 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:52.159015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:52.159187 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:52.159312 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.159318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.160127 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.160132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.160836 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.160850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.161305 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.161307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.161372 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.161863 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:52.161886 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:52.163650 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":80,"threads_started":1}
W20260810 07:15:52.163911 28464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:52.158597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47725 (local address 127.12.117.190:42513)
0810 07:15:52.158813 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:52.158817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:52.158846 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:15:52.158879 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:52.158881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:52.158921 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:52.158985 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:52.159386 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.160005 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.160967 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.161216 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.161297 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.163778 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:47725: BlockingRecv error: recv got EOF from 127.12.117.129:47725 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":67,"threads_started":1}
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:52.337734 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:52.332529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:56369)
0810 07:15:52.332801 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:52.332813 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:52.332924 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:52.333173 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:52.333176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:52.333367 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:52.333465 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.333471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.334368 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.334371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.334927 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.334935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.335398 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.335401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.335488 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.335909 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:52.335920 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:52.337619 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
W20260810 07:15:52.337888 28464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:52.332595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:56369 (local address 127.12.117.190:42513)
0810 07:15:52.332732 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:52.332736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:52.332880 (+ 144us) server_negotiation.cc:407] Connection header received
0810 07:15:52.333009 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:52.333013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:52.333062 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:52.333126 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:52.333581 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.334263 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.335041 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.335300 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.335391 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.337767 (+ 2376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:56369: BlockingRecv error: recv got EOF from 127.12.117.130:56369 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:52 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:52.531126 28469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:52.525735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:55401)
0810 07:15:52.526003 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:52.526016 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:52.526141 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:52.526389 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:52.526392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:52.526591 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:52.526691 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.526697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.527474 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.527477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.528041 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:52.528050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.528488 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.528492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:52.528582 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.529033 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:52.529044 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:52.531011 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":70,"threads_started":1}
W20260810 07:15:52.531291 28464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:52.525800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:55401 (local address 127.12.117.190:42513)
0810 07:15:52.525929 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:52.525934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:52.526085 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:15:52.526229 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:52.526232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:52.526281 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:52.526349 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:52.526807 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.527371 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.528168 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:52.528390 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:52.528473 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:52.531148 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:55401: BlockingRecv error: recv got EOF from 127.12.117.131:55401 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:53.169579 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:53.164458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:34291)
0810 07:15:53.164727 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:53.164740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:53.164829 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:53.165055 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:53.165058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:53.165232 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:53.165329 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.165334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.166105 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.166108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.166662 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.166670 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.167055 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.167057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.167116 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.167576 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:53.167587 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:53.169452 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":86,"threads_started":1}
W20260810 07:15:53.169709 28474 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:53.164543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:34291 (local address 127.12.117.190:42513)
0810 07:15:53.164784 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:53.164787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:53.164800 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:53.164903 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:53.164905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:53.164953 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:53.165028 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:53.165411 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.165982 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.166748 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.166973 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.167045 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.169577 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:34291: BlockingRecv error: recv got EOF from 127.12.117.129:34291 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:53.343814 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:53.338437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:60793)
0810 07:15:53.338713 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:53.338727 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:53.338823 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:53.339036 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:53.339038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:53.339216 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:53.339350 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.339358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.340108 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.340110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.340669 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.340676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.341212 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.341216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.341290 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.341831 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:53.341846 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:53.343630 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
W20260810 07:15:53.343850 28474 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:53.338513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:60793 (local address 127.12.117.190:42513)
0810 07:15:53.338630 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:53.338634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:53.338787 (+ 153us) server_negotiation.cc:407] Connection header received
0810 07:15:53.338900 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:53.338902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:53.338951 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:53.339008 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:53.339438 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.340022 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.340782 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.341063 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.341172 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.343754 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:60793: BlockingRecv error: recv got EOF from 127.12.117.130:60793 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:53 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:53.537443 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:53.531908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:32997)
0810 07:15:53.532309 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:53.532324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:53.532442 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:53.532716 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:53.532719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:53.532913 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:53.533028 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.533037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.533872 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.533876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.534451 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:53.534460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.534903 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.534906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:53.534993 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.535479 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:53.535490 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:53.537307 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":107,"threads_started":1}
W20260810 07:15:53.537573 28474 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:53.531986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:32997 (local address 127.12.117.190:42513)
0810 07:15:53.532128 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:53.532134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:53.532399 (+ 265us) server_negotiation.cc:407] Connection header received
0810 07:15:53.532531 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:53.532534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:53.532583 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:53.532665 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:53.533155 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.533761 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.534571 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:53.534798 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:53.534886 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:53.537456 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:32997: BlockingRecv error: recv got EOF from 127.12.117.131:32997 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:54.175766 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:54.170256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:33217)
0810 07:15:54.170539 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:54.170554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:54.170648 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:54.170852 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:54.170855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:54.171029 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:54.171129 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.171153 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.172108 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.172112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.172754 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.172767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.173201 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.173203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.173287 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.173802 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:54.173816 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:54.175640 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
W20260810 07:15:54.175886 28478 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:54.170314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:33217 (local address 127.12.117.190:42513)
0810 07:15:54.170591 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:54.170594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:54.170608 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:54.170715 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:54.170717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:54.170757 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:54.170828 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:54.171285 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.171978 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.172868 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.173096 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.173186 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.175769 (+ 2583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:33217: BlockingRecv error: recv got EOF from 127.12.117.129:33217 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":123,"threads_started":1}
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:54.349516 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:54.344570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:51131)
0810 07:15:54.344810 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:54.344823 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:54.344922 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:54.345123 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:54.345125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:54.345331 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:54.345456 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.345463 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.346144 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.346147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.346708 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.346715 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.347104 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.347106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.347167 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.347640 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:54.347652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:54.349399 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":70,"threads_started":1}
W20260810 07:15:54.349618 28478 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:54.344647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:51131 (local address 127.12.117.190:42513)
0810 07:15:54.344761 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:54.344765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:54.344879 (+ 114us) server_negotiation.cc:407] Connection header received
0810 07:15:54.344991 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:54.344993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:54.345034 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:54.345100 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:54.345540 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.346052 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.346807 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.347026 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.347094 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.349518 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:51131: BlockingRecv error: recv got EOF from 127.12.117.130:51131 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:54 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:54.543397 28490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:54.538202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:46945)
0810 07:15:54.538459 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:54.538473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:54.538569 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:54.538776 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:54.538778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:54.538952 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:54.539050 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.539056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.539744 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.539747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.540306 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:54.540315 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.540676 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.540678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:54.540737 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.541223 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:54.541236 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:54.543240 (+ 2004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":68,"threads_started":1}
W20260810 07:15:54.543484 28478 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:54.538329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:46945 (local address 127.12.117.190:42513)
0810 07:15:54.538451 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:54.538455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:54.538532 (+ 77us) server_negotiation.cc:407] Connection header received
0810 07:15:54.538639 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:54.538641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:54.538685 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:54.538747 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:54.539130 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.539673 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.540393 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:54.540593 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:54.540668 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:54.543389 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:46945: BlockingRecv error: recv got EOF from 127.12.117.131:46945 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:55.181702 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:55.176474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:45791)
0810 07:15:55.176731 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:55.176744 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:55.176843 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:55.177059 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:55.177061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:55.177238 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:55.177336 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.177342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.178087 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.178091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.178752 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.178765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.179293 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.179298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.179379 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.179844 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:55.179855 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:55.181586 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":72,"threads_started":1}
W20260810 07:15:55.181839 28492 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:55.176538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45791 (local address 127.12.117.190:42513)
0810 07:15:55.176781 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:55.176785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:55.176798 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:15:55.176918 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:55.176920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:55.176963 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:55.177034 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:55.177419 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.177942 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.178857 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.179161 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.179240 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.181720 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:45791: BlockingRecv error: recv got EOF from 127.12.117.129:45791 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:55.355225 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:55.350207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:42927)
0810 07:15:55.350450 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:55.350464 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:55.350563 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:55.350782 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:55.350784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:55.351017 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:55.351141 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.351148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.351884 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.351886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.352457 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.352465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.352861 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.352864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.352923 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.353356 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:55.353366 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:55.355091 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
W20260810 07:15:55.355376 28492 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:55.350268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:42927 (local address 127.12.117.190:42513)
0810 07:15:55.350406 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:55.350410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:55.350523 (+ 113us) server_negotiation.cc:407] Connection header received
0810 07:15:55.350630 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:55.350632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:55.350674 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:55.350741 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:55.351246 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.351793 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.352559 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.352779 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.352853 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.355233 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:42927: BlockingRecv error: recv got EOF from 127.12.117.130:42927 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:55 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:55.549909 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:55.544146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:54007)
0810 07:15:55.544382 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:55.544395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:55.544505 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:55.544771 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:55.544775 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:55.544983 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:55.545084 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.545090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.545861 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.545865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.546436 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:55.546445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.546885 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.546889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:55.546974 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.547508 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:55.547526 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:55.549704 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":65,"threads_started":1}
W20260810 07:15:55.549966 28492 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:55.544198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:54007 (local address 127.12.117.190:42513)
0810 07:15:55.544316 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:55.544321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:55.544464 (+ 143us) server_negotiation.cc:407] Connection header received
0810 07:15:55.544604 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:55.544617 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:55.544665 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:55.544730 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:55.545208 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.545759 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.546553 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:55.546785 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:55.546873 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:55.549865 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:54007: BlockingRecv error: recv got EOF from 127.12.117.131:54007 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:56.187194 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:56.182389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:43743)
0810 07:15:56.182645 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:56.182658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:56.182747 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:56.182963 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:56.182964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:56.183153 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:56.183253 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.183283 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.183969 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.183972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.184526 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.184533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.184893 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.184896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.184954 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.185408 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:56.185418 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:56.187057 (+ 1639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
W20260810 07:15:56.187351 28499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:56.182457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:43743 (local address 127.12.117.190:42513)
0810 07:15:56.182678 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:56.182682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:56.182696 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:56.182821 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:56.182822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:56.182864 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:56.182937 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:56.183365 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.183890 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.184605 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.184817 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.184885 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.187203 (+ 2318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:43743: BlockingRecv error: recv got EOF from 127.12.117.129:43743 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":72,"threads_started":1}
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:56.360785 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:56.355923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:43761)
0810 07:15:56.356198 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:56.356211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:56.356310 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:56.356512 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:56.356514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:56.356692 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:56.356788 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.356794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.357473 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.357475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.358040 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.358048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.358404 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.358406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.358465 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.358891 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:56.358902 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:56.360648 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
W20260810 07:15:56.360863 28499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:56.355996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:43761 (local address 127.12.117.190:42513)
0810 07:15:56.356115 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:56.356119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:56.356270 (+ 151us) server_negotiation.cc:407] Connection header received
0810 07:15:56.356373 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:56.356375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:56.356415 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:56.356487 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:56.356870 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.357378 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.358121 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.358329 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.358395 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.360765 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:43761: BlockingRecv error: recv got EOF from 127.12.117.130:43761 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:56 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:56.558735 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:56.550736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:39587)
0810 07:15:56.550991 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:56.551005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:56.551104 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:56.551334 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:56.551336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:56.551509 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:56.551607 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.551613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.552294 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.552296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.552843 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:56.552849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.553222 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.553224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:56.553282 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.556735 (+ 3453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:56.556750 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:56.558621 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
W20260810 07:15:56.558887 28499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:56.550790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:39587 (local address 127.12.117.190:42513)
0810 07:15:56.550915 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:56.550920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:56.551064 (+ 144us) server_negotiation.cc:407] Connection header received
0810 07:15:56.551184 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:56.551186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:56.551231 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:56.551320 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:56.551693 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.552217 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.552929 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:56.553144 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:56.553217 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:56.558775 (+ 5558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:39587: BlockingRecv error: recv got EOF from 127.12.117.131:39587 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:57.193289 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:57.187995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:50233)
0810 07:15:57.188263 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:57.188277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:57.188378 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:57.188665 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:57.188668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:57.188851 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:57.188969 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.188976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.189794 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.189797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.190391 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.190401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.190802 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.190804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.190862 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.191405 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:57.191417 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:57.193178 (+ 1761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":89,"threads_started":1}
W20260810 07:15:57.193434 28506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:57.188043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:50233 (local address 127.12.117.190:42513)
0810 07:15:57.188282 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:57.188286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:57.188340 (+ 54us) server_negotiation.cc:407] Connection header received
0810 07:15:57.188467 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:57.188470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:57.188516 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:57.188594 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:57.189069 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.189665 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.190488 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.190718 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.190793 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.193302 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:50233: BlockingRecv error: recv got EOF from 127.12.117.129:50233 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":75,"threads_started":1}
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:57.366858 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:57.361468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:56607)
0810 07:15:57.361850 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:57.361876 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:57.361987 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:57.362248 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:57.362251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:57.362481 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:57.362588 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.362595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.363358 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.363362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.363956 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.363965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.364406 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.364410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.364495 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.364968 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:57.364979 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:57.366701 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":76,"threads_started":1}
W20260810 07:15:57.366945 28506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:57.361629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:56607 (local address 127.12.117.190:42513)
0810 07:15:57.361756 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:57.361761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:57.361947 (+ 186us) server_negotiation.cc:407] Connection header received
0810 07:15:57.362079 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:57.362082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:57.362140 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:57.362204 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:57.362704 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.363238 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.364072 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.364305 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.364391 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.366844 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:56607: BlockingRecv error: recv got EOF from 127.12.117.130:56607 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:57 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:57.564379 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:57.559433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:53855)
0810 07:15:57.559691 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:57.559704 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:57.559803 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:57.560002 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:57.560004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:57.560176 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:57.560272 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.560277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.560954 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.560956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.561527 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:57.561534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.561908 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.561910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:57.561971 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.562397 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:57.562407 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:57.564255 (+ 1848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
W20260810 07:15:57.564499 28506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:57.559487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:53855 (local address 127.12.117.190:42513)
0810 07:15:57.559628 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:57.559633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:57.559766 (+ 133us) server_negotiation.cc:407] Connection header received
0810 07:15:57.559867 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:57.559869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:57.559915 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:57.559976 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:57.560353 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.560875 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.561612 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:57.561824 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:57.561902 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:57.564386 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:53855: BlockingRecv error: recv got EOF from 127.12.117.131:53855 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:58.199306 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:58.194013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:57477)
0810 07:15:58.194297 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:58.194314 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:58.194412 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:58.194637 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:58.194639 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:58.194822 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:58.194919 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.194925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.195729 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.195733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.196301 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.196310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.196786 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.196789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.196847 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.197346 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:58.197358 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:58.199148 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
W20260810 07:15:58.199440 28521 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:58.194077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:57477 (local address 127.12.117.190:42513)
0810 07:15:58.194405 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:58.194408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:58.194422 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:15:58.194477 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:58.194479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:58.194529 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:58.194600 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:58.195032 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.195622 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.196451 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.196688 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.196771 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.199311 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:57477: BlockingRecv error: recv got EOF from 127.12.117.129:57477 (error 108)
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":79,"threads_started":1}
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:58.373209 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:58.367587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:33791)
0810 07:15:58.367860 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:58.367873 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:58.367980 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:58.368221 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:58.368224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:58.368423 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:58.368530 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.368537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.369339 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.369343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.370095 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.370108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.370644 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.370648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.370746 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.371291 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:58.371304 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:58.373081 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":83,"threads_started":1}
W20260810 07:15:58.373315 28521 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:58.367680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:33791 (local address 127.12.117.190:42513)
0810 07:15:58.367819 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:58.367824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:58.367937 (+ 113us) server_negotiation.cc:407] Connection header received
0810 07:15:58.368071 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:58.368075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:58.368121 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:58.368190 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:58.368644 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.369231 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.370211 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.370525 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.370639 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.373214 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:33791: BlockingRecv error: recv got EOF from 127.12.117.130:33791 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:58 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.131@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:58.573776 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:58.566289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.131:60171)
0810 07:15:58.567853 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:58.567866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:58.567970 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:58.568198 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:58.568201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:58.568400 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:58.568508 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.568514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.569431 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.569436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.570233 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:58.570245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.570827 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.570831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:58.570918 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.571472 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:58.571485 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:58.573607 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1493,"thread_start_us":1409,"threads_started":1}
W20260810 07:15:58.573845 28521 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:58.566310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:60171 (local address 127.12.117.190:42513)
0810 07:15:58.566425 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:58.566428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:58.567934 (+ 1506us) server_negotiation.cc:407] Connection header received
0810 07:15:58.568038 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:58.568041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:58.568097 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:58.568179 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:58.568630 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.569312 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.570343 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:58.570705 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:58.570797 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:58.573740 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.131:60171: BlockingRecv error: recv got EOF from 127.12.117.131:60171 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:15:59 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:59 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.129@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:59.205289 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:59.200088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.129:47179)
0810 07:15:59.200343 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:59.200355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:59.200438 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:59.200652 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:59.200655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:59.200831 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:59.200929 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.200934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.201634 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.201636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.202199 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.202206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.202677 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.202681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.202774 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.203298 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:59.203316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:59.205143 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
W20260810 07:15:59.205400 28528 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:59.200178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47179 (local address 127.12.117.190:42513)
0810 07:15:59.200390 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:59.200401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:59.200413 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:15:59.200516 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:59.200518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:59.200561 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:59.200624 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:59.201019 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.201554 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.202280 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.202573 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.202651 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.205280 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.129:47179: BlockingRecv error: recv got EOF from 127.12.117.129:47179 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":65,"threads_started":1}
Aug 10 07:15:59 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 07:15:59 dist-test-slave-gjrz krb5kdc[26655](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.117.130@KRBTEST.COM for oryx/127.12.117.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 07:15:59.379184 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:59.374017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:42513 (local address 127.12.117.130:59401)
0810 07:15:59.374286 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:59.374299 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:59.374402 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:59.374643 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:59.374645 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:59.374845 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:59.374943 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.374950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.375741 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.375745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.376318 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.376326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.376762 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.376765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.376847 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.377291 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:15:59.377302 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:59.379057 (+ 1755us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.117.190:42513: Server oryx/127.12.117.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
W20260810 07:15:59.379330 28528 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:15:59.374084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:59401 (local address 127.12.117.190:42513)
0810 07:15:59.374222 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:59.374227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:59.374364 (+ 137us) server_negotiation.cc:407] Connection header received
0810 07:15:59.374488 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:59.374492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:59.374538 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:59.374604 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:59.375059 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.375633 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.376433 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.376662 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.376749 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.379191 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.117.130:59401: BlockingRecv error: recv got EOF from 127.12.117.130:59401 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:15:59.460136 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 27671
I20260810 07:15:59.474761 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 27940
I20260810 07:15:59.483471 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 28171
I20260810 07:15:59.491905 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 27532
2026-08-10T07:15:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18071 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T07:15:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:15:59Z Disabled control of system clock
I20260810 07:15:59.525161 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38103
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:38103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 07:15:59.630672 28539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:59.630880 28539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:59.630913 28539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:59.634768 28539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:15:59.634817 28539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:59.634832 28539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:59.634846 28539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:15:59.634871 28539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:15:59.639411 28539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:38103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:59.640256 28539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:59.641434 28539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:15:59.649482 28560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:59.649487 28550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:59.649567 28539 server_base.cc:1034] running on GCE node
W20260810 07:15:59.649540 28553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:59.650254 28539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:59.651218 28539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:59.652384 28539 hybrid_clock.cc:648] HybridClock initialized: now 1786346159652370 us; error 28 us; skew 500 ppm
I20260810 07:15:59.654070 28539 webserver.cc:460] Webserver started at http://127.12.117.190:34895/ using document root <none> and password file <none>
I20260810 07:15:59.654598 28539 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:59.654662 28539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:59.654856 28539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:59.656800 28539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "0e89ce9bd1384150bd3e707f15cf306f"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "75172c5de2a26d752921a4a275270036"
server_key_iv: "f24d1bcba0d4e55e397d2863aac34f1c"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.657318 28539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "0e89ce9bd1384150bd3e707f15cf306f"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "75172c5de2a26d752921a4a275270036"
server_key_iv: "f24d1bcba0d4e55e397d2863aac34f1c"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.660696 28539 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:15:59.662906 28571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:59.664065 28539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:15:59.664186 28539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "0e89ce9bd1384150bd3e707f15cf306f"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "75172c5de2a26d752921a4a275270036"
server_key_iv: "f24d1bcba0d4e55e397d2863aac34f1c"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.664275 28539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:59.675042 28539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:59.675602 28539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:59.675745 28539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:59.682080 28539 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:38103
I20260810 07:15:59.682098 28671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:38103 every 8 connection(s)
I20260810 07:15:59.682971 28539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:15:59.685938 28674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:15:59.689872 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 28539
I20260810 07:15:59.689965 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:15:59.690222 12758 external_mini_cluster.cc:1428] Setting key 5f3d0677c888475f030b8e885f0d2a1c
I20260810 07:15:59.691427 28674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f: Bootstrap starting.
I20260810 07:15:59.694021 28674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f: Neither blocks nor log segments found. Creating new log.
I20260810 07:15:59.694885 28674 log.cc:824] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:15:59.696771 28674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f: No bootstrap required, opened a new log
I20260810 07:15:59.700218 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:59.691592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.12.117.190:38103)
0810 07:15:59.692259 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:59.692279 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:59.692320 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:15:59.693132 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:59.693156 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:59.693625 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:59.693989 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:59.694444 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.695767 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.696664 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.697339 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.697492 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.698259 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:59.698266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:59.698283 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:59.698762 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:59.698833 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:59.698859 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:59.698959 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:59.699032 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:15:59.699326 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":108,"threads_started":1}
I20260810 07:15:59.701246 28674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } }
I20260810 07:15:59.701499 28674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:15:59.701557 28674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e89ce9bd1384150bd3e707f15cf306f, State: Initialized, Role: FOLLOWER
I20260810 07:15:59.702011 28674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } }
I20260810 07:15:59.702164 28674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:15:59.702229 28674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:15:59.702327 28674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:15:59.703138 28674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } }
I20260810 07:15:59.703465 28674 leader_election.cc:302] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e89ce9bd1384150bd3e707f15cf306f; no voters:
I20260810 07:15:59.703730 28674 leader_election.cc:288] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:15:59.703811 28679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:15:59.704039 28679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [term 1 LEADER]: Becoming Leader. State: Replica: 0e89ce9bd1384150bd3e707f15cf306f, State: Running, Role: LEADER
I20260810 07:15:59.704358 28679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } }
I20260810 07:15:59.704825 28674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:15:59.705808 28681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e89ce9bd1384150bd3e707f15cf306f. Latest consensus state: current_term: 1 leader_uuid: "0e89ce9bd1384150bd3e707f15cf306f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } } }
I20260810 07:15:59.705935 28681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:59.706204 28680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e89ce9bd1384150bd3e707f15cf306f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e89ce9bd1384150bd3e707f15cf306f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38103 } } }
I20260810 07:15:59.706300 28680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f [sys.catalog]: This master's current role is: LEADER
I20260810 07:15:59.706385 28693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:15:59.709218 28693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:15:59.714593 28693 catalog_manager.cc:1347] Generated new cluster ID: bba17edbe7f34977bc5acebcf69d5dfe
I20260810 07:15:59.714663 28693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:15:59.735584 28693 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:15:59.736404 28693 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:15:59.741974 28693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e89ce9bd1384150bd3e707f15cf306f: Generated new TSK 0
I20260810 07:15:59.742543 28693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:15:59.747325 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--builtin_ntp_servers=127.12.117.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 07:15:59.855576 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:15:59.855794 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:15:59.855829 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:15:59.859795 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:15:59.859848 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:15:59.859930 28712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:15:59.864658 28712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:15:59.865516 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:15:59.866711 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:15:59.875660 28712 server_base.cc:1034] running on GCE node
W20260810 07:15:59.875605 28727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:59.875648 28721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:15:59.875612 28723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:15:59.876246 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:15:59.876813 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:15:59.877996 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786346159877955 us; error 31 us; skew 500 ppm
I20260810 07:15:59.879716 28712 webserver.cc:460] Webserver started at http://127.12.117.129:37347/ using document root <none> and password file <none>
I20260810 07:15:59.880227 28712 fs_manager.cc:359] Metadata directory not provided
I20260810 07:15:59.880291 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:15:59.880468 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:15:59.882366 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "dd31bbab81624ae49adf7d4821a4bd43"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "c6be2f3b1dc6e4d7e1c473c87f7a3ebf"
server_key_iv: "5fb40838b16ed9d122a14cee03098b35"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.882872 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "dd31bbab81624ae49adf7d4821a4bd43"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "c6be2f3b1dc6e4d7e1c473c87f7a3ebf"
server_key_iv: "5fb40838b16ed9d122a14cee03098b35"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.886211 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:15:59.888460 28748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:59.889520 28712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:15:59.889643 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "dd31bbab81624ae49adf7d4821a4bd43"
format_stamp: "Formatted at 2026-08-10 07:15:59 on dist-test-slave-gjrz"
server_key: "c6be2f3b1dc6e4d7e1c473c87f7a3ebf"
server_key_iv: "5fb40838b16ed9d122a14cee03098b35"
server_key_version: "encryptionkey@0"
I20260810 07:15:59.889730 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:15:59.920292 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:15:59.920859 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:15:59.921030 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:15:59.921535 28712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:15:59.922389 28712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:15:59.922436 28712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:59.922485 28712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:15:59.922515 28712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:15:59.930615 28712 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:39327
I20260810 07:15:59.930680 28956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:39327 every 8 connection(s)
I20260810 07:15:59.931478 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:15:59.933604 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 28712
I20260810 07:15:59.933712 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:15:59.933985 12758 external_mini_cluster.cc:1428] Setting key ec94051137eccefdcbee59e255501495
I20260810 07:15:59.935962 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--builtin_ntp_servers=127.12.117.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:15:59.941236 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:59.933098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:37483 (local address 127.12.117.190:38103)
0810 07:15:59.933221 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:15:59.933224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:15:59.934294 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 07:15:59.935767 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:15:59.935771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:15:59.935826 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:15:59.935922 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:15:59.937047 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.937742 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.939365 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.939573 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.939656 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.940793 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:15:59.940796 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:15:59.940798 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:15:59.940811 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:15:59.940831 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:15:59.940838 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:15:59.940888 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:15:59.941068 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:15:59.941120 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:15:59.942327 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:15:59.933429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38103 (local address 127.12.117.129:37483)
0810 07:15:59.934131 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 07:15:59.934181 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:15:59.935550 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:15:59.936155 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:15:59.936165 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:15:59.936267 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:15:59.936854 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.936871 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.937871 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.937875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.939207 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:15:59.939220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:15:59.939722 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:15:59.939726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:15:59.939904 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:15:59.940482 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:15:59.940484 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:15:59.940537 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:15:59.940620 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:15:59.940910 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:15:59.940919 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:15:59.940926 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 07:15:59.941077 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:15:59.941368 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260810 07:15:59.943481 28958 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38103
I20260810 07:15:59.943701 28958 heartbeater.cc:464] Registering TS with master...
I20260810 07:15:59.944171 28958 heartbeater.cc:511] Master 127.12.117.190:38103 requested a full tablet report, sending...
I20260810 07:15:59.945560 28602 ts_manager.cc:194] Registered new tserver with Master: dd31bbab81624ae49adf7d4821a4bd43 (127.12.117.129:39327)
I20260810 07:15:59.946653 28602 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.129:37483
W20260810 07:16:00.046293 28964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:00.046506 28964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:00.046542 28964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:00.050419 28964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:00.050464 28964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:00.050524 28964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:16:00.055022 28964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:00.055897 28964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:00.057119 28964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:16:00.065864 28964 server_base.cc:1034] running on GCE node
W20260810 07:16:00.065786 28975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.065819 28973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.065788 28982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.066329 28964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:00.066869 28964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:00.068022 28964 hybrid_clock.cc:648] HybridClock initialized: now 1786346160068006 us; error 29 us; skew 500 ppm
I20260810 07:16:00.069710 28964 webserver.cc:460] Webserver started at http://127.12.117.130:42387/ using document root <none> and password file <none>
I20260810 07:16:00.070212 28964 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:00.070278 28964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:00.070472 28964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:00.072388 28964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "9c467be627104dffb36cd40f05a71087"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "b4ca680fc8ac78bce6f9cd1a60a2905c"
server_key_iv: "58f2b9ffa88734a316e64c7271d12965"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.072908 28964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "9c467be627104dffb36cd40f05a71087"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "b4ca680fc8ac78bce6f9cd1a60a2905c"
server_key_iv: "58f2b9ffa88734a316e64c7271d12965"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.076220 28964 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:16:00.078485 28993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.079598 28964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:16:00.079712 28964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "9c467be627104dffb36cd40f05a71087"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "b4ca680fc8ac78bce6f9cd1a60a2905c"
server_key_iv: "58f2b9ffa88734a316e64c7271d12965"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.079802 28964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:00.104362 28964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:00.104936 28964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:00.105108 28964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:00.105659 28964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:00.106503 28964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:00.106550 28964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.106598 28964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:00.106627 28964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.114496 28964 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44737
I20260810 07:16:00.114521 29216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44737 every 8 connection(s)
I20260810 07:16:00.115283 28964 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:16:00.122833 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 28964
I20260810 07:16:00.122934 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:16:00.123286 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.116892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:37675 (local address 127.12.117.190:38103)
0810 07:16:00.117014 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.117018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.117972 (+ 954us) server_negotiation.cc:407] Connection header received
0810 07:16:00.119234 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.119237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.119303 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.119371 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.120129 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.120642 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.121664 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.121863 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.121940 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:00.122830 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:00.122833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:00.122835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:00.122846 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:00.122864 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:00.122872 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:00.122940 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:00.123102 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:16:00.123153 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:16:00.123288 12758 external_mini_cluster.cc:1428] Setting key 9ee04225e2865296ccd3e7304a88ba76
I20260810 07:16:00.124161 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.117236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38103 (local address 127.12.117.130:37675)
0810 07:16:00.117830 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:00.117867 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:00.119067 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:00.119508 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:00.119515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:00.119592 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:00.120022 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:00.120034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.120727 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.120730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:00.121578 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:00.121587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.121972 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.121974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:00.122097 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:00.122556 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:16:00.122557 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:00.122604 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:16:00.122683 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:00.122953 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:00.122961 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:00.122967 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:16:00.123091 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 07:16:00.123384 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260810 07:16:00.125299 29217 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38103
I20260810 07:16:00.125322 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--builtin_ntp_servers=127.12.117.148:43181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 07:16:00.125554 29217 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:00.126056 29217 heartbeater.cc:511] Master 127.12.117.190:38103 requested a full tablet report, sending...
I20260810 07:16:00.127024 28602 ts_manager.cc:194] Registered new tserver with Master: 9c467be627104dffb36cd40f05a71087 (127.12.117.130:44737)
I20260810 07:16:00.127597 28602 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.130:37675
W20260810 07:16:00.231822 29221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:00.232030 29221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:00.232062 29221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:00.235956 29221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:00.235996 29221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:00.236052 29221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:16:00.240545 29221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:43181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:00.241381 29221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:00.242691 29221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:00.251392 29233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.251408 29235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.251444 29221 server_base.cc:1034] running on GCE node
W20260810 07:16:00.251398 29231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.251936 29221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:00.252493 29221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:00.253644 29221 hybrid_clock.cc:648] HybridClock initialized: now 1786346160253621 us; error 36 us; skew 500 ppm
I20260810 07:16:00.255223 29221 webserver.cc:460] Webserver started at http://127.12.117.131:33673/ using document root <none> and password file <none>
I20260810 07:16:00.255736 29221 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:00.255802 29221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:00.255999 29221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:00.257834 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "66b4d8f6e3e14f8993f6acc90944c45f"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "24d495bf5a8414fa716b45c359bb4443"
server_key_iv: "13cc577afc6a2558b02b3db5eab039b2"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.258349 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "66b4d8f6e3e14f8993f6acc90944c45f"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "24d495bf5a8414fa716b45c359bb4443"
server_key_iv: "13cc577afc6a2558b02b3db5eab039b2"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.261715 29221 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260810 07:16:00.263958 29249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.265045 29221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:16:00.265156 29221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "66b4d8f6e3e14f8993f6acc90944c45f"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "24d495bf5a8414fa716b45c359bb4443"
server_key_iv: "13cc577afc6a2558b02b3db5eab039b2"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.265244 29221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:00.276674 29221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:00.277211 29221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:00.277359 29221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:00.277842 29221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:00.278650 29221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:00.278697 29221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.278744 29221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:00.278774 29221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.287039 29221 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:33501
I20260810 07:16:00.287060 29473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:33501 every 8 connection(s)
I20260810 07:16:00.287829 29221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:16:00.291543 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 29221
I20260810 07:16:00.291625 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:16:00.291898 12758 external_mini_cluster.cc:1428] Setting key 0efebf9570ae3ed05b416fe973916e69
I20260810 07:16:00.296366 28677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.289396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:57035 (local address 127.12.117.190:38103)
0810 07:16:00.289547 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.289552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.290484 (+ 932us) server_negotiation.cc:407] Connection header received
0810 07:16:00.291824 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.291828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.291886 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.291976 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.292964 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.293615 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.294788 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.295028 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.295110 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:00.296004 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:00.296007 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:00.296009 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:00.296022 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:00.296042 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:00.296050 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:00.296105 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:00.296208 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:16:00.296259 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:16:00.297063 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.289753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38103 (local address 127.12.117.131:57035)
0810 07:16:00.290347 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:00.290383 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:00.291614 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:00.292128 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:00.292136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:00.292226 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:00.292831 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:00.292847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.293758 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.293761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:00.294679 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:00.294689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:00.295140 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:00.295142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:00.295305 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:00.295774 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:16:00.295776 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:00.295811 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:16:00.295869 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:00.296111 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:00.296116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:00.296120 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:16:00.296198 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:16:00.296391 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260810 07:16:00.298000 29476 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38103
I20260810 07:16:00.298199 29476 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:00.298660 29476 heartbeater.cc:511] Master 127.12.117.190:38103 requested a full tablet report, sending...
I20260810 07:16:00.299568 28602 ts_manager.cc:194] Registered new tserver with Master: 66b4d8f6e3e14f8993f6acc90944c45f (127.12.117.131:33501)
I20260810 07:16:00.300108 28602 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.131:57035
I20260810 07:16:00.305363 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:16:00.307818 28677 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:16:00.307114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.12.117.190:38103)
0810 07:16:00.307212 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.307215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.307388 (+ 173us) server_negotiation.cc:407] Connection header received
0810 07:16:00.307470 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.307472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.307507 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.307567 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.307729 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43836: BlockingRecv error: recv got EOF from 127.0.0.1:43836 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:16:00.308207 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 28712
I20260810 07:16:00.325904 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 28964
I20260810 07:16:00.334465 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 29221
I20260810 07:16:00.345000 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 28539
2026-08-10T07:16:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (851 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T07:16:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:16:00Z Disabled control of system clock
I20260810 07:16:00.376335 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45809
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:45809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:16:00.482344 29493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:00.482565 29493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:00.482599 29493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:00.486527 29493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:16:00.486575 29493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:00.486590 29493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:00.486605 29493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:16:00.486618 29493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:16:00.491230 29493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:45809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:00.492113 29493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:00.493314 29493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:00.501041 29509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.501075 29504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.501082 29503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.501387 29493 server_base.cc:1034] running on GCE node
I20260810 07:16:00.501891 29493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:00.502846 29493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:00.504004 29493 hybrid_clock.cc:648] HybridClock initialized: now 1786346160503989 us; error 26 us; skew 500 ppm
I20260810 07:16:00.505657 29493 webserver.cc:460] Webserver started at http://127.12.117.190:42439/ using document root <none> and password file <none>
I20260810 07:16:00.506155 29493 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:00.506213 29493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:00.506397 29493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:00.508278 29493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "7d2edd9b2fcd46728afd6cf629c73c81"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "4d7bf51c8684530f1d9f49ba4a505296"
server_key_iv: "16d66948f232d0380135e9803d480334"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.508793 29493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "7d2edd9b2fcd46728afd6cf629c73c81"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "4d7bf51c8684530f1d9f49ba4a505296"
server_key_iv: "16d66948f232d0380135e9803d480334"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.512142 29493 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:16:00.514277 29523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.515391 29493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:16:00.515498 29493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "7d2edd9b2fcd46728afd6cf629c73c81"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "4d7bf51c8684530f1d9f49ba4a505296"
server_key_iv: "16d66948f232d0380135e9803d480334"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.515585 29493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:00.519717 29493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:00.520247 29493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:00.520394 29493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:00.526607 29493 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:45809
I20260810 07:16:00.526654 29624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:45809 every 8 connection(s)
I20260810 07:16:00.527441 29493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:16:00.530485 29625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:00.531186 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 29493
I20260810 07:16:00.531311 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:16:00.531570 12758 external_mini_cluster.cc:1428] Setting key 6751df36acae792537b56390607a78bc
I20260810 07:16:00.536033 29625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81: Bootstrap starting.
I20260810 07:16:00.536576 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.532840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36202 (local address 127.12.117.190:45809)
0810 07:16:00.533393 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.533407 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.533439 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:16:00.534049 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.534065 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.534380 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.534635 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.535409 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:00.535414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:00.535426 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:00.535437 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:00.535490 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:00.535506 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:00.535613 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:00.535709 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:16:00.535935 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":97,"threads_started":1}
I20260810 07:16:00.538184 29625 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:00.538874 29625 log.cc:824] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:00.540555 29625 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81: No bootstrap required, opened a new log
I20260810 07:16:00.543730 29625 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } }
I20260810 07:16:00.543933 29625 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:00.543973 29625 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2edd9b2fcd46728afd6cf629c73c81, State: Initialized, Role: FOLLOWER
I20260810 07:16:00.544306 29625 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } }
I20260810 07:16:00.544418 29625 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:16:00.544461 29625 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:16:00.544525 29625 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:00.545229 29625 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } }
I20260810 07:16:00.545504 29625 leader_election.cc:302] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d2edd9b2fcd46728afd6cf629c73c81; no voters:
I20260810 07:16:00.545727 29625 leader_election.cc:288] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:16:00.545831 29635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:00.546026 29635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [term 1 LEADER]: Becoming Leader. State: Replica: 7d2edd9b2fcd46728afd6cf629c73c81, State: Running, Role: LEADER
I20260810 07:16:00.546331 29635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } }
I20260810 07:16:00.546694 29625 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:16:00.547791 29638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d2edd9b2fcd46728afd6cf629c73c81. Latest consensus state: current_term: 1 leader_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } } }
I20260810 07:16:00.547915 29638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:00.548177 29637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2edd9b2fcd46728afd6cf629c73c81" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45809 } } }
I20260810 07:16:00.548273 29637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81 [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:00.548847 29651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:16:00.551348 29651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:16:00.556349 29651 catalog_manager.cc:1347] Generated new cluster ID: 58c1d94961d245ed9746badd9f70d735
I20260810 07:16:00.556406 29651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:16:00.575824 29651 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:16:00.576119 29651 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:16:00.583123 29651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d2edd9b2fcd46728afd6cf629c73c81: Generated new TSK 0
I20260810 07:16:00.583710 29651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:16:00.592548 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--builtin_ntp_servers=127.12.117.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:16:00.698833 29674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:00.699051 29674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:00.699095 29674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:00.702986 29674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:00.703030 29674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:00.703090 29674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:16:00.707679 29674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:00.708549 29674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:00.709750 29674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:00.718654 29689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.718667 29682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.718730 29674 server_base.cc:1034] running on GCE node
W20260810 07:16:00.718807 29692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.719246 29674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:00.719820 29674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:00.720968 29674 hybrid_clock.cc:648] HybridClock initialized: now 1786346160720952 us; error 27 us; skew 500 ppm
I20260810 07:16:00.722591 29674 webserver.cc:460] Webserver started at http://127.12.117.129:34635/ using document root <none> and password file <none>
I20260810 07:16:00.723071 29674 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:00.723136 29674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:00.723337 29674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:00.725255 29674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "6e142ac7110240729ad34e5b07ac5004"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "65ce724ee0abe83ad174bb69dbb96dbb"
server_key_iv: "0504ccb0e68561c0c2291682fd5d749c"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.725732 29674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "6e142ac7110240729ad34e5b07ac5004"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "65ce724ee0abe83ad174bb69dbb96dbb"
server_key_iv: "0504ccb0e68561c0c2291682fd5d749c"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.729086 29674 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:16:00.731351 29709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.732451 29674 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:16:00.732558 29674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "6e142ac7110240729ad34e5b07ac5004"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "65ce724ee0abe83ad174bb69dbb96dbb"
server_key_iv: "0504ccb0e68561c0c2291682fd5d749c"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.732656 29674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:00.736861 29674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:00.737401 29674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:00.737542 29674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:00.738018 29674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:00.738860 29674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:00.738906 29674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.738945 29674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:00.738973 29674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.746871 29674 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:44509
I20260810 07:16:00.746953 29921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:44509 every 8 connection(s)
I20260810 07:16:00.747720 29674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:16:00.748636 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 29674
I20260810 07:16:00.748723 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:16:00.749018 12758 external_mini_cluster.cc:1428] Setting key 4fe45864ca81c210fb5e9143f1934791
I20260810 07:16:00.750939 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--builtin_ntp_servers=127.12.117.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 07:16:00.753551 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.749423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:56093 (local address 127.12.117.190:45809)
0810 07:16:00.749661 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.749666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.750696 (+ 1030us) server_negotiation.cc:407] Connection header received
0810 07:16:00.752002 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.752006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.752046 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.752117 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.753134 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:00.753138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:00.753140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:00.753152 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:00.753172 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:00.753180 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:00.753250 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:00.753373 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 07:16:00.753424 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 07:16:00.754295 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.749869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45809 (local address 127.12.117.129:56093)
0810 07:16:00.750571 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:00.750609 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:00.751801 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:00.752282 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:00.752289 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:00.752372 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:00.752876 (+ 504us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:16:00.752877 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:00.752935 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:16:00.753002 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:00.753246 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:00.753254 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:00.753260 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:16:00.753407 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:16:00.753628 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260810 07:16:00.755403 29922 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45809
I20260810 07:16:00.755613 29922 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:00.755963 29922 heartbeater.cc:511] Master 127.12.117.190:45809 requested a full tablet report, sending...
I20260810 07:16:00.757344 29560 ts_manager.cc:194] Registered new tserver with Master: 6e142ac7110240729ad34e5b07ac5004 (127.12.117.129:44509)
W20260810 07:16:00.858589 29925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:00.858811 29925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:00.858865 29925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:00.862723 29925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:00.862772 29925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:00.862845 29925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:16:00.867393 29925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:00.868287 29925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:00.869467 29925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:00.878077 29932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:00.878158 29935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.878293 29925 server_base.cc:1034] running on GCE node
W20260810 07:16:00.878305 29937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:00.878748 29925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:00.879349 29925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:00.880503 29925 hybrid_clock.cc:648] HybridClock initialized: now 1786346160880482 us; error 33 us; skew 500 ppm
I20260810 07:16:00.882138 29925 webserver.cc:460] Webserver started at http://127.12.117.130:32805/ using document root <none> and password file <none>
I20260810 07:16:00.882632 29925 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:00.882694 29925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:00.882867 29925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:00.884786 29925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "c81f0d194c6b40d6b3b49f516c54d027"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "932b429428b077d2ae8c902309e69a8c"
server_key_iv: "e9947c9fbf609f0bbbde1396ad0b0171"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.885298 29925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "c81f0d194c6b40d6b3b49f516c54d027"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "932b429428b077d2ae8c902309e69a8c"
server_key_iv: "e9947c9fbf609f0bbbde1396ad0b0171"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.888968 29925 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:16:00.891175 29950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.892274 29925 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:16:00.892376 29925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "c81f0d194c6b40d6b3b49f516c54d027"
format_stamp: "Formatted at 2026-08-10 07:16:00 on dist-test-slave-gjrz"
server_key: "932b429428b077d2ae8c902309e69a8c"
server_key_iv: "e9947c9fbf609f0bbbde1396ad0b0171"
server_key_version: "encryptionkey@0"
I20260810 07:16:00.892468 29925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:00.896615 29925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:00.897181 29925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:00.897330 29925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:00.897827 29925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:00.898659 29925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:00.898706 29925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.898754 29925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:00.898783 29925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:00.906741 29925 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44803
I20260810 07:16:00.906764 30169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44803 every 8 connection(s)
I20260810 07:16:00.907588 29925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:16:00.912948 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.909165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:57409 (local address 127.12.117.190:45809)
0810 07:16:00.909273 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:00.909276 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:00.910224 (+ 948us) server_negotiation.cc:407] Connection header received
0810 07:16:00.911462 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:00.911466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:00.911513 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:00.911582 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:00.912529 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:00.912533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:00.912536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:00.912551 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:00.912569 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:00.912578 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:00.912651 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:00.912770 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:16:00.912814 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:16:00.913681 30177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:00.909473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45809 (local address 127.12.117.130:57409)
0810 07:16:00.910088 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:00.910124 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:00.911212 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:00.911743 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:00.911750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:00.911830 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:00.912291 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:16:00.912293 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:00.912328 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:16:00.912391 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:00.912645 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:00.912653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:00.912658 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:16:00.912803 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 07:16:00.913048 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":105,"threads_started":1}
I20260810 07:16:00.914683 30173 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45809
I20260810 07:16:00.914880 30173 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:00.915226 30173 heartbeater.cc:511] Master 127.12.117.190:45809 requested a full tablet report, sending...
I20260810 07:16:00.916170 29560 ts_manager.cc:194] Registered new tserver with Master: c81f0d194c6b40d6b3b49f516c54d027 (127.12.117.130:44803)
I20260810 07:16:00.917305 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 29925
I20260810 07:16:00.917414 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:16:00.917675 12758 external_mini_cluster.cc:1428] Setting key b90168be029a5df884a6ba0923ccb0a6
I20260810 07:16:00.919322 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--builtin_ntp_servers=127.12.117.148:37773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 07:16:01.026119 30178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:01.026336 30178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:01.026371 30178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:01.030242 30178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:01.030289 30178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:01.030342 30178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:16:01.034845 30178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:37773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:01.035804 30178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:01.037153 30178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:01.046109 30194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:01.046144 30186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:01.046144 30178 server_base.cc:1034] running on GCE node
W20260810 07:16:01.046135 30189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:01.046806 30178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:01.047382 30178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:01.048574 30178 hybrid_clock.cc:648] HybridClock initialized: now 1786346161048539 us; error 45 us; skew 500 ppm
I20260810 07:16:01.050186 30178 webserver.cc:460] Webserver started at http://127.12.117.131:44517/ using document root <none> and password file <none>
I20260810 07:16:01.050688 30178 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:01.050745 30178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:01.050906 30178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:01.052840 30178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "516bd61d732a4779b74367338f2151c3"
format_stamp: "Formatted at 2026-08-10 07:16:01 on dist-test-slave-gjrz"
server_key: "67d7209ebe359360b0b8070b5af9983e"
server_key_iv: "13df317c4092db4e7e0f39d7a7d1030c"
server_key_version: "encryptionkey@0"
I20260810 07:16:01.053335 30178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "516bd61d732a4779b74367338f2151c3"
format_stamp: "Formatted at 2026-08-10 07:16:01 on dist-test-slave-gjrz"
server_key: "67d7209ebe359360b0b8070b5af9983e"
server_key_iv: "13df317c4092db4e7e0f39d7a7d1030c"
server_key_version: "encryptionkey@0"
I20260810 07:16:01.056692 30178 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 07:16:01.059027 30208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:01.060163 30178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:16:01.060269 30178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "516bd61d732a4779b74367338f2151c3"
format_stamp: "Formatted at 2026-08-10 07:16:01 on dist-test-slave-gjrz"
server_key: "67d7209ebe359360b0b8070b5af9983e"
server_key_iv: "13df317c4092db4e7e0f39d7a7d1030c"
server_key_version: "encryptionkey@0"
I20260810 07:16:01.060359 30178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:01.064656 30178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:01.065196 30178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:01.065344 30178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:01.065834 30178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:01.066792 30178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:01.066849 30178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:01.066901 30178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:01.066929 30178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:01.074945 30178 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:45935
I20260810 07:16:01.074998 30389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:45935 every 8 connection(s)
I20260810 07:16:01.075757 30178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:16:01.081259 29632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:01.077366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:51673 (local address 127.12.117.190:45809)
0810 07:16:01.077482 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:01.077485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:01.078409 (+ 924us) server_negotiation.cc:407] Connection header received
0810 07:16:01.079690 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:01.079694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:01.079741 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:01.079813 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:01.080828 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:01.080831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:01.080834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:16:01.080848 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:01.080869 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:16:01.080877 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:01.080954 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:01.081077 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 07:16:01.081125 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:16:01.082008 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:01.077673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45809 (local address 127.12.117.131:51673)
0810 07:16:01.078269 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:01.078306 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:01.079476 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:01.079988 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:01.079996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:01.080080 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:01.080572 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:16:01.080574 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:01.080610 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:16:01.080674 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:01.080946 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:01.080954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:01.080960 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:16:01.081108 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 07:16:01.081331 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":110,"threads_started":1}
I20260810 07:16:01.083105 30392 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45809
I20260810 07:16:01.083333 30392 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:01.083705 30392 heartbeater.cc:511] Master 127.12.117.190:45809 requested a full tablet report, sending...
I20260810 07:16:01.084688 29560 ts_manager.cc:194] Registered new tserver with Master: 516bd61d732a4779b74367338f2151c3 (127.12.117.131:45935)
I20260810 07:16:01.085454 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 30178
I20260810 07:16:01.085538 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:16:01.085779 12758 external_mini_cluster.cc:1428] Setting key 4dfd0ab4941fb94a9a922d2170d3b214
I20260810 07:16:01.087574 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 07:16:01.090106 29632 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:16:01.089295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36210 (local address 127.12.117.190:45809)
0810 07:16:01.089395 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:01.089399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:01.089575 (+ 176us) server_negotiation.cc:407] Connection header received
0810 07:16:01.089679 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:01.089682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:01.089712 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:01.089772 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:01.089988 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36210: BlockingRecv error: recv got EOF from 127.0.0.1:36210 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:16:01.090262 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 29674
I20260810 07:16:01.117345 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 29925
I20260810 07:16:01.127218 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 30178
I20260810 07:16:01.134177 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 29493
2026-08-10T07:16:01Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (791 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:16:01 dist-test-slave-gjrz krb5kdc[30417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:16:01 dist-test-slave-gjrz krb5kdc[30417](info): set up 2 sockets
Aug 10 07:16:01 dist-test-slave-gjrz krb5kdc[30417](info): commencing operation
krb5kdc: starting...
W20260810 07:16:02.382284 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:16:02 dist-test-slave-gjrz krb5kdc[30417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:16:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:16:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:16:02.493763 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39157
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39157
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:02.597982 30446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:02.598206 30446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:02.598244 30446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:02.602362 30446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:16:02.602409 30446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:02.602427 30446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:02.602442 30446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:16:02.602455 30446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:16:02.607340 30446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39157
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:02.608192 30446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:02.609009 30446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:02.616624 30455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:02.616719 30446 server_base.cc:1034] running on GCE node
W20260810 07:16:02.616624 30458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:02.616621 30454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:02.617455 30446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:02.618397 30446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:02.619544 30446 hybrid_clock.cc:648] HybridClock initialized: now 1786346162619525 us; error 27 us; skew 500 ppm
Aug 10 07:16:02 dist-test-slave-gjrz krb5kdc[30417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346162, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:02.624676 30446 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:16:02.625460 30446 webserver.cc:460] Webserver started at http://127.12.117.190:34661/ using document root <none> and password file <none>
I20260810 07:16:02.625955 30446 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:02.626022 30446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:02.626211 30446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:02.628120 30446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "deb645eae0be4ceda06bb15762868295"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "be4ac5de6ed456db5ac76693f92ccfbc"
server_key_iv: "ce254f2b7210e9d20fa921b01a0a5ff9"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.628648 30446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "deb645eae0be4ceda06bb15762868295"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "be4ac5de6ed456db5ac76693f92ccfbc"
server_key_iv: "ce254f2b7210e9d20fa921b01a0a5ff9"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.632126 30446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:16:02.634279 30468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:02.635401 30446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:16:02.635519 30446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "deb645eae0be4ceda06bb15762868295"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "be4ac5de6ed456db5ac76693f92ccfbc"
server_key_iv: "ce254f2b7210e9d20fa921b01a0a5ff9"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.635609 30446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:02.661670 30446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:02.664326 30446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:02.664500 30446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:02.670840 30446 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39157
I20260810 07:16:02.671078 30559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39157 every 8 connection(s)
I20260810 07:16:02.671768 30446 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:16:02.674382 30562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:02.678866 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 30446
I20260810 07:16:02.678972 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:16:02.679219 12758 external_mini_cluster.cc:1428] Setting key 9460eff444fe7cf170ed4cb9d306e596
I20260810 07:16:02.679786 30562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: Bootstrap starting.
I20260810 07:16:02.681957 30562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:02.682672 30562 log.cc:824] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:02.684499 30562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: No bootstrap required, opened a new log
Aug 10 07:16:02 dist-test-slave-gjrz krb5kdc[30417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:02.688191 30562 raft_consensus.cc:348] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } }
I20260810 07:16:02.688467 30562 raft_consensus.cc:374] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:02.688537 30562 raft_consensus.cc:729] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deb645eae0be4ceda06bb15762868295, State: Initialized, Role: FOLLOWER
I20260810 07:16:02.688983 30562 consensus_queue.cc:260] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } }
I20260810 07:16:02.689112 30562 raft_consensus.cc:388] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:16:02.689173 30562 raft_consensus.cc:482] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:16:02.689270 30562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:02.690280 30562 raft_consensus.cc:504] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } }
I20260810 07:16:02.690620 30562 leader_election.cc:302] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: deb645eae0be4ceda06bb15762868295; no voters:
I20260810 07:16:02.690946 30562 leader_election.cc:288] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:16:02.691035 30569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:02.691293 30569 raft_consensus.cc:686] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [term 1 LEADER]: Becoming Leader. State: Replica: deb645eae0be4ceda06bb15762868295, State: Running, Role: LEADER
I20260810 07:16:02.691599 30569 consensus_queue.cc:237] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } }
I20260810 07:16:02.692209 30562 sys_catalog.cc:564] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:16:02.692763 30582 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader deb645eae0be4ceda06bb15762868295. Latest consensus state: current_term: 1 leader_uuid: "deb645eae0be4ceda06bb15762868295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } } }
I20260810 07:16:02.693496 30582 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:02.693529 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:02.680559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.12.117.190:39157)
0810 07:16:02.680940 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:02.680961 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:02.680997 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:16:02.681593 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:02.681609 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:02.681925 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:02.682238 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:02.683024 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.684081 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.684950 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.685403 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.685567 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:02.688162 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:02.688168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:02.688184 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:02.688215 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:02.690202 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:02.690658 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:02.690664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:02.690670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:02.690808 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:02.691190 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:02.691194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:02.691195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:02.691760 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:02.691951 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:02.692290 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 07:16:02.692578 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":107,"threads_started":1}
I20260810 07:16:02.693383 30579 sys_catalog.cc:455] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "deb645eae0be4ceda06bb15762868295" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "deb645eae0be4ceda06bb15762868295" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39157 } } }
I20260810 07:16:02.694468 30579 sys_catalog.cc:458] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295 [sys.catalog]: This master's current role is: LEADER
W20260810 07:16:02.696982 30600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:16:02.697052 30600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:16:02.697108 30598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:16:02.697945 30598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:16:02.702889 30598 catalog_manager.cc:1347] Generated new cluster ID: 32dc1806156548f7a296ce61bfaca27c
I20260810 07:16:02.702940 30598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:16:02.707442 30598 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:16:02.708235 30598 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:16:02.719564 30598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: Generated new TSK 0
I20260810 07:16:02.720239 30598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:16:02.753758 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:02.861100 30613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:02.861320 30613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:02.861364 30613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:02.865314 30613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:02.865363 30613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:02.865430 30613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:16:02.870433 30613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:02.871352 30613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:02.872287 30613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:02.881273 30628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:02.881275 30623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:02.881273 30624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:02.881369 30613 server_base.cc:1034] running on GCE node
I20260810 07:16:02.881953 30613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:02.882547 30613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:02.883697 30613 hybrid_clock.cc:648] HybridClock initialized: now 1786346162883672 us; error 40 us; skew 500 ppm
Aug 10 07:16:02 dist-test-slave-gjrz krb5kdc[30417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346162, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:02.886252 30613 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:16:02.887033 30613 webserver.cc:460] Webserver started at http://127.12.117.129:36343/ using document root <none> and password file <none>
I20260810 07:16:02.887584 30613 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:02.887655 30613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:02.887854 30613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:02.889820 30613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "f45a6b3554e84a23a79f77d83503f02f"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "251fa26a8da739529d6ed100cfc0c8fb"
server_key_iv: "6292c801e1912a0fe92cb25fe0af07c7"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.890355 30613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "f45a6b3554e84a23a79f77d83503f02f"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "251fa26a8da739529d6ed100cfc0c8fb"
server_key_iv: "6292c801e1912a0fe92cb25fe0af07c7"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.894055 30613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:16:02.896327 30639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:02.897420 30613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:16:02.897529 30613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "f45a6b3554e84a23a79f77d83503f02f"
format_stamp: "Formatted at 2026-08-10 07:16:02 on dist-test-slave-gjrz"
server_key: "251fa26a8da739529d6ed100cfc0c8fb"
server_key_iv: "6292c801e1912a0fe92cb25fe0af07c7"
server_key_version: "encryptionkey@0"
I20260810 07:16:02.897624 30613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:02.913610 30613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:02.916285 30613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:02.916479 30613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:02.917068 30613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:02.917863 30613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:02.917910 30613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:02.917961 30613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:02.917989 30613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:02.926015 30613 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:43167
I20260810 07:16:02.926064 30839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:43167 every 8 connection(s)
I20260810 07:16:02.926990 30613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:16:02.929046 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 30613
I20260810 07:16:02.929150 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:16:02.929481 12758 external_mini_cluster.cc:1428] Setting key 0f358840a78d1378b744fb2ae5eae2d1
Aug 10 07:16:02 dist-test-slave-gjrz krb5kdc[30417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346162, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:16:02.940295 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:02.928654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:36769 (local address 127.12.117.190:39157)
0810 07:16:02.928807 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:02.928811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:02.929690 (+ 879us) server_negotiation.cc:407] Connection header received
0810 07:16:02.931113 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:02.931116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:02.931159 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:02.931217 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:02.932647 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.933188 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.934317 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.934551 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.934650 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:02.936998 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:02.937001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:02.937003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:02.937026 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:02.938305 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:02.938791 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:02.938794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:02.938795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:02.938873 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:02.939217 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:02.939220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:02.939222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:02.939444 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:02.939533 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:02.940009 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 07:16:02.940134 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:16:02.941138 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:02.929016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39157 (local address 127.12.117.129:36769)
0810 07:16:02.929532 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:02.929588 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:02.930877 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:02.931427 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:02.931436 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:02.931885 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:02.932513 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:02.932532 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.933321 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.933324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:02.934195 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:02.934206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:02.934689 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:02.934694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:02.934857 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:02.935442 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:02.935463 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:02.936837 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:02.938405 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:02.938411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:02.938426 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:02.938673 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:02.938955 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:02.938958 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:02.938960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:02.939091 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:02.939544 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:02.939551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:02.939780 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 07:16:02.939946 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 07:16:02.940301 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:16:02.942330 30840 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39157
I20260810 07:16:02.942569 30840 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:02.943104 30840 heartbeater.cc:511] Master 127.12.117.190:39157 requested a full tablet report, sending...
I20260810 07:16:02.944706 30490 ts_manager.cc:194] Registered new tserver with Master: f45a6b3554e84a23a79f77d83503f02f (127.12.117.129:43167)
I20260810 07:16:02.945850 30490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:36769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:16:02.956283 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:03.062335 30847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:03.062548 30847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:03.062589 30847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:03.066452 30847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:03.066498 30847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:03.066563 30847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:16:03.071491 30847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:03.072343 30847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:03.073235 30847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:16:03.082222 30847 server_base.cc:1034] running on GCE node
W20260810 07:16:03.082116 30853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:03.082139 30858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:03.082139 30855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:03.082787 30847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:03.083410 30847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:03.084568 30847 hybrid_clock.cc:648] HybridClock initialized: now 1786346163084554 us; error 26 us; skew 500 ppm
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[30417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346163, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:03.087131 30847 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:16:03.087942 30847 webserver.cc:460] Webserver started at http://127.12.117.130:34831/ using document root <none> and password file <none>
I20260810 07:16:03.088452 30847 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:03.088511 30847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:03.088758 30847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:03.090684 30847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "18a35e4031174d4b987facd451db7179"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "6d206da03ae4faa13eba9f8541d27326"
server_key_iv: "b13b5c8dee430a9c25592a81dffea134"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.091228 30847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "18a35e4031174d4b987facd451db7179"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "6d206da03ae4faa13eba9f8541d27326"
server_key_iv: "b13b5c8dee430a9c25592a81dffea134"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.094928 30847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:16:03.097236 30867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.098362 30847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:16:03.098459 30847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "18a35e4031174d4b987facd451db7179"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "6d206da03ae4faa13eba9f8541d27326"
server_key_iv: "b13b5c8dee430a9c25592a81dffea134"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.098553 30847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:03.119499 30847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:03.122205 30847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:03.122365 30847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:03.122897 30847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:03.123714 30847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:03.123765 30847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.123805 30847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:03.123833 30847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.132037 30847 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:41401
I20260810 07:16:03.132072 31057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:41401 every 8 connection(s)
I20260810 07:16:03.132851 30847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[30417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346163, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:03.141723 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 30847
I20260810 07:16:03.141834 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:16:03.142148 12758 external_mini_cluster.cc:1428] Setting key 470a478a10ced08b1490b5af6bf8590c
I20260810 07:16:03.145788 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.134478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:60845 (local address 127.12.117.190:39157)
0810 07:16:03.134635 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.134639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.135456 (+ 817us) server_negotiation.cc:407] Connection header received
0810 07:16:03.136730 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.136733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.136782 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.136857 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:03.137918 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.138502 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.139713 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.139936 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.140011 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.142318 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:03.142321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:03.142323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:03.142347 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:03.143879 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.144348 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.144352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.144354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.144447 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.144749 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.144752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.144754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.144959 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:03.145054 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.145500 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.145632 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:16:03.146502 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.134819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39157 (local address 127.12.117.130:60845)
0810 07:16:03.135325 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.135367 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.136541 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.137032 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.137039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.137355 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:03.137803 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.137817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.138640 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.138644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.139587 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.139598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.140074 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.140077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.140209 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.140707 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:03.140724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:03.142140 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:03.143987 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:03.143992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:03.144003 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:03.144255 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:03.144540 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:03.144542 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:03.144543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:03.144673 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:03.145042 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:03.145047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:03.145268 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.145430 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.145730 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260810 07:16:03.147703 31059 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39157
I20260810 07:16:03.147946 31059 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:03.148525 31059 heartbeater.cc:511] Master 127.12.117.190:39157 requested a full tablet report, sending...
I20260810 07:16:03.149641 30490 ts_manager.cc:194] Registered new tserver with Master: 18a35e4031174d4b987facd451db7179 (127.12.117.130:41401)
I20260810 07:16:03.150400 30490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:60845
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:16:03.168278 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:03.275247 31073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:03.275473 31073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:03.275527 31073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:03.279377 31073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:03.279423 31073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:03.279488 31073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:16:03.284360 31073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39157
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:03.285209 31073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:03.286078 31073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:03.295033 31081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:03.295071 31083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:03.295038 31086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:03.295205 31073 server_base.cc:1034] running on GCE node
I20260810 07:16:03.295719 31073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:03.296279 31073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:03.297429 31073 hybrid_clock.cc:648] HybridClock initialized: now 1786346163297399 us; error 47 us; skew 500 ppm
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[30417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346163, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:03.299893 31073 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:16:03.300732 31073 webserver.cc:460] Webserver started at http://127.12.117.131:42983/ using document root <none> and password file <none>
I20260810 07:16:03.301252 31073 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:03.301318 31073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:03.301523 31073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:16:03.303834 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "f289d4e00362462eacaa0adad05402a0"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "57638e6d720538140e362369d9e34116"
server_key_iv: "c262a775f1028e7754dfe0b49e402ec8"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.304718 31073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "f289d4e00362462eacaa0adad05402a0"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "57638e6d720538140e362369d9e34116"
server_key_iv: "c262a775f1028e7754dfe0b49e402ec8"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.308183 31073 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:16:03.310432 31097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.311594 31073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:16:03.311704 31073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "f289d4e00362462eacaa0adad05402a0"
format_stamp: "Formatted at 2026-08-10 07:16:03 on dist-test-slave-gjrz"
server_key: "57638e6d720538140e362369d9e34116"
server_key_iv: "c262a775f1028e7754dfe0b49e402ec8"
server_key_version: "encryptionkey@0"
I20260810 07:16:03.311791 31073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:03.326116 31073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:03.328729 31073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:03.328887 31073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:03.329536 31073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:03.330387 31073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:03.330446 31073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.330502 31073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:03.330536 31073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:03.338920 31073 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:38721
I20260810 07:16:03.338977 31275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:38721 every 8 connection(s)
I20260810 07:16:03.339811 31073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:16:03.343391 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 31073
I20260810 07:16:03.343492 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:16:03.343765 12758 external_mini_cluster.cc:1428] Setting key 7d49a447582f123e241c0943f3c96b3c
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[30417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346163, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:03.352430 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.341421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:46175 (local address 127.12.117.190:39157)
0810 07:16:03.341561 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.341565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.342388 (+ 823us) server_negotiation.cc:407] Connection header received
0810 07:16:03.343816 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.343820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.343876 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.343946 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:03.345251 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.345841 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.346956 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.347176 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.347287 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.349542 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:03.349545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:03.349547 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:03.349571 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:03.350673 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.351142 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.351144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.351145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.351208 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.351567 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.351569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.351570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.351730 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:03.351802 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.352231 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.352318 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:16:03.353116 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.341770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39157 (local address 127.12.117.131:46175)
0810 07:16:03.342251 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.342290 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.343619 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.344130 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.344137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.344581 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:03.345126 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.345142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.345956 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.345959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.346841 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.346851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.347326 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.347331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.347494 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.347996 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:03.348011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:03.349392 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:03.350770 (+ 1378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:03.350775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:03.350785 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:03.351061 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:03.351334 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:03.351337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:03.351338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:03.351476 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:03.351802 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:03.351807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:03.352043 (+ 236us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.352190 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.352418 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":157,"threads_started":1}
I20260810 07:16:03.354058 31276 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39157
I20260810 07:16:03.354261 31276 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:03.354681 31276 heartbeater.cc:511] Master 127.12.117.190:39157 requested a full tablet report, sending...
I20260810 07:16:03.355589 30490 ts_manager.cc:194] Registered new tserver with Master: f289d4e00362462eacaa0adad05402a0 (127.12.117.131:38721)
I20260810 07:16:03.356178 30490 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:46175
I20260810 07:16:03.357175 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:16:03.365162 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.358880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54510 (local address 127.12.117.190:39157)
0810 07:16:03.359025 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.359029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.359177 (+ 148us) server_negotiation.cc:407] Connection header received
0810 07:16:03.359314 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.359317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.359353 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.359409 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:03.360023 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.360486 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.361288 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.361487 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.361576 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.362683 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:03.362687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:03.362690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:03.362713 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:03.363723 (+ 1010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.364095 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.364098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.364100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.364170 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:03.364406 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:03.364410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:03.364411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:03.364611 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:03.364726 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.364939 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.365034 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 07:16:03.367457 30490 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:16:03.370230 30490 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:16:03.384115 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:41401 (local address 127.0.0.1:51082)
0810 07:16:03.379057 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.379107 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.379237 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.379880 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.379884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.379901 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:03.379992 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.379999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381394 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.381397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.382622 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.382632 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.383628 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.383632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.383739 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.383776 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.383872 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.383981 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":61,"spinlock_wait_cycles":6784,"thread_start_us":79,"threads_started":1}
I20260810 07:16:03.384156 31300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.12.117.130:41401)
0810 07:16:03.379518 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.379526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.379545 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:16:03.379601 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.379604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.379742 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.379882 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:03.380106 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381261 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.382745 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.383494 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.383630 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.383870 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.383922 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.384009 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":125,"threads_started":1}
I20260810 07:16:03.384778 31292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:43167 (local address 127.0.0.1:58516)
0810 07:16:03.379058 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.379109 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.379238 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.379798 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.379804 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.379826 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:03.379944 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.379951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381425 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.381428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.382897 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.382909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.384121 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.384125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.384466 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.384487 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.384557 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.384626 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":129,"spinlock_wait_cycles":2176,"thread_start_us":65,"threads_started":1}
I20260810 07:16:03.384861 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58516 (local address 127.12.117.129:43167)
0810 07:16:03.378627 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.378642 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.379192 (+ 550us) server_negotiation.cc:407] Connection header received
0810 07:16:03.379479 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.379485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.379647 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.379767 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:03.380084 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381257 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.383019 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.383826 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.383937 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.384055 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.384610 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.384724 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":128,"threads_started":1}
I20260810 07:16:03.386925 30952 tablet_service.cc:1467] Processing CreateTablet for tablet fe29862b4aee4570b57c660f1ea18034 (DEFAULT_TABLE table=test-table [id=d3ab15d7e88549f78c89a3f64b06e65d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:03.387392 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38721 (local address 127.0.0.1:47768)
0810 07:16:03.379976 (+ 1560us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.379989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.380072 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.380441 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.380444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.380459 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:03.380536 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.380542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381425 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.381428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.382900 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.382913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.384341 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.384345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.384428 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.384450 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.384513 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.384577 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1506,"mutex_wait_us":147,"thread_start_us":78,"threads_started":1}
I20260810 07:16:03.387676 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.378749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.12.117.131:38721)
0810 07:16:03.379084 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.379095 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.380153 (+ 1058us) server_negotiation.cc:407] Connection header received
0810 07:16:03.380210 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.380213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.380328 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.380444 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:03.380619 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.381305 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.383347 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.384040 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.384159 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.384735 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.384781 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.384858 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":89,"threads_started":1}
I20260810 07:16:03.387379 30725 tablet_service.cc:1467] Processing CreateTablet for tablet fe29862b4aee4570b57c660f1ea18034 (DEFAULT_TABLE table=test-table [id=d3ab15d7e88549f78c89a3f64b06e65d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:03.388015 30952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe29862b4aee4570b57c660f1ea18034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:03.388401 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe29862b4aee4570b57c660f1ea18034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:03.390340 31169 tablet_service.cc:1467] Processing CreateTablet for tablet fe29862b4aee4570b57c660f1ea18034 (DEFAULT_TABLE table=test-table [id=d3ab15d7e88549f78c89a3f64b06e65d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:03.391296 31169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe29862b4aee4570b57c660f1ea18034. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:03.394958 31306 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Bootstrap starting.
I20260810 07:16:03.397192 31306 tablet_bootstrap.cc:654] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:03.397650 31302 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Bootstrap starting.
I20260810 07:16:03.397883 31307 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Bootstrap starting.
I20260810 07:16:03.397984 31306 log.cc:824] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:03.399569 31302 tablet_bootstrap.cc:654] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:03.399868 31307 tablet_bootstrap.cc:654] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:03.400606 31302 log.cc:824] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:03.400995 31306 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: No bootstrap required, opened a new log
I20260810 07:16:03.401046 31307 log.cc:824] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:03.401194 31306 ts_tablet_manager.cc:1397] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:16:03.402372 31302 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: No bootstrap required, opened a new log
I20260810 07:16:03.402549 31302 ts_tablet_manager.cc:1397] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:16:03.403203 31307 tablet_bootstrap.cc:492] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: No bootstrap required, opened a new log
I20260810 07:16:03.403401 31307 ts_tablet_manager.cc:1397] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:16:03.404822 31306 raft_consensus.cc:348] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.405076 31306 raft_consensus.cc:374] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:03.405135 31306 raft_consensus.cc:729] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18a35e4031174d4b987facd451db7179, State: Initialized, Role: FOLLOWER
I20260810 07:16:03.405357 31302 raft_consensus.cc:348] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.405576 31302 raft_consensus.cc:374] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:03.405615 31302 raft_consensus.cc:729] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45a6b3554e84a23a79f77d83503f02f, State: Initialized, Role: FOLLOWER
I20260810 07:16:03.405579 31306 consensus_queue.cc:260] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.406028 31302 consensus_queue.cc:260] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.406487 31306 ts_tablet_manager.cc:1428] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:16:03.406826 31059 heartbeater.cc:503] Master 127.12.117.190:39157 was elected leader, sending a full tablet report...
I20260810 07:16:03.406916 31307 raft_consensus.cc:348] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.407136 31307 raft_consensus.cc:374] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:03.407191 31307 raft_consensus.cc:729] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f289d4e00362462eacaa0adad05402a0, State: Initialized, Role: FOLLOWER
I20260810 07:16:03.407528 31302 ts_tablet_manager.cc:1428] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:16:03.407562 31307 consensus_queue.cc:260] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.407855 30840 heartbeater.cc:503] Master 127.12.117.190:39157 was elected leader, sending a full tablet report...
I20260810 07:16:03.408138 31276 heartbeater.cc:503] Master 127.12.117.190:39157 was elected leader, sending a full tablet report...
I20260810 07:16:03.408438 31307 ts_tablet_manager.cc:1428] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260810 07:16:03.428040 30841 tablet.cc:2369] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:16:03.544667 31313 raft_consensus.cc:482] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:16:03.544863 31313 raft_consensus.cc:504] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.545948 31313 leader_election.cc:288] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f45a6b3554e84a23a79f77d83503f02f (127.12.117.129:43167), f289d4e00362462eacaa0adad05402a0 (127.12.117.131:38721)
I20260810 07:16:03.549636 31065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.546168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:43167 (local address 127.12.117.130:53005)
0810 07:16:03.546399 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.546415 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.546526 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.546815 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.546818 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.546835 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:03.546967 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.546975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.547762 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.547765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.548577 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.548586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.549224 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.549228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.549329 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.549355 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.549442 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.549503 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260810 07:16:03.549675 31296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.546264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:53005 (local address 127.12.117.129:43167)
0810 07:16:03.546404 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.546409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.546490 (+ 81us) server_negotiation.cc:407] Connection header received
0810 07:16:03.546640 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.546644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.546700 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.546777 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:03.547079 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.547658 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.548696 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.549115 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.549206 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.549247 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.549457 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.549546 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:16:03.549769 31298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.546347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:50091 (local address 127.12.117.131:38721)
0810 07:16:03.546453 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.546457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.546724 (+ 267us) server_negotiation.cc:407] Connection header received
0810 07:16:03.546798 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.546801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.546853 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.546983 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:03.547222 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.547741 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.548825 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.549253 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.549360 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.549405 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.549587 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.549660 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:16:03.549809 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.546215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38721 (local address 127.12.117.130:50091)
0810 07:16:03.546605 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.546618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:03.546721 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:03.546988 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:03.546990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:03.547004 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:03.547120 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.547127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.547846 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.547848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.548707 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:03.548717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.549369 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.549372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:03.549467 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.549489 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:16:03.549561 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 07:16:03.549626 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":127,"threads_started":1}
I20260810 07:16:03.550267 31199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe29862b4aee4570b57c660f1ea18034" candidate_uuid: "18a35e4031174d4b987facd451db7179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f289d4e00362462eacaa0adad05402a0" is_pre_election: true
I20260810 07:16:03.550266 30762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe29862b4aee4570b57c660f1ea18034" candidate_uuid: "18a35e4031174d4b987facd451db7179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f45a6b3554e84a23a79f77d83503f02f" is_pre_election: true
I20260810 07:16:03.550563 31199 raft_consensus.cc:2454] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a35e4031174d4b987facd451db7179 in term 0.
I20260810 07:16:03.550563 30762 raft_consensus.cc:2454] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18a35e4031174d4b987facd451db7179 in term 0.
I20260810 07:16:03.551005 30868 leader_election.cc:302] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a35e4031174d4b987facd451db7179, f45a6b3554e84a23a79f77d83503f02f; no voters:
I20260810 07:16:03.551298 31313 raft_consensus.cc:2781] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:16:03.551388 31313 raft_consensus.cc:482] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:16:03.551450 31313 raft_consensus.cc:3037] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:03.552419 31313 raft_consensus.cc:504] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.552697 31313 leader_election.cc:288] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [CANDIDATE]: Term 1 election: Requested vote from peers f45a6b3554e84a23a79f77d83503f02f (127.12.117.129:43167), f289d4e00362462eacaa0adad05402a0 (127.12.117.131:38721)
I20260810 07:16:03.553031 31199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe29862b4aee4570b57c660f1ea18034" candidate_uuid: "18a35e4031174d4b987facd451db7179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f289d4e00362462eacaa0adad05402a0"
I20260810 07:16:03.553053 30762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe29862b4aee4570b57c660f1ea18034" candidate_uuid: "18a35e4031174d4b987facd451db7179" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f45a6b3554e84a23a79f77d83503f02f"
I20260810 07:16:03.553133 31199 raft_consensus.cc:3037] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:03.553162 30762 raft_consensus.cc:3037] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:03.554078 31199 raft_consensus.cc:2454] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a35e4031174d4b987facd451db7179 in term 1.
I20260810 07:16:03.554136 30762 raft_consensus.cc:2454] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18a35e4031174d4b987facd451db7179 in term 1.
I20260810 07:16:03.554332 30871 leader_election.cc:302] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18a35e4031174d4b987facd451db7179, f289d4e00362462eacaa0adad05402a0; no voters:
I20260810 07:16:03.554536 31313 raft_consensus.cc:2781] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:03.554731 31313 raft_consensus.cc:686] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 1 LEADER]: Becoming Leader. State: Replica: 18a35e4031174d4b987facd451db7179, State: Running, Role: LEADER
I20260810 07:16:03.555037 31313 consensus_queue.cc:237] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } }
I20260810 07:16:03.558038 30489 catalog_manager.cc:5495] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18a35e4031174d4b987facd451db7179 (127.12.117.130). New cstate: current_term: 1 leader_uuid: "18a35e4031174d4b987facd451db7179" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18a35e4031174d4b987facd451db7179" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 41401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 } health_report { overall_health: UNKNOWN } } }
I20260810 07:16:03.568565 31300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.565320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.12.117.130:41401)
0810 07:16:03.565438 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:03.565443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:03.565456 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:16:03.565588 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:03.565591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:03.565638 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:03.565721 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:16:03.565968 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.566426 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.567235 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:03.567493 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:03.567579 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:03.567851 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:16:03.568215 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:16:03.568317 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:03.568387 (+ 70us) server_negotiation.cc:299] Negotiation successful
0810 07:16:03.568436 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:16:03.573608 30762 raft_consensus.cc:1261] T fe29862b4aee4570b57c660f1ea18034 P f45a6b3554e84a23a79f77d83503f02f [term 1 FOLLOWER]: Refusing update from remote peer 18a35e4031174d4b987facd451db7179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:03.573609 31199 raft_consensus.cc:1261] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 1 FOLLOWER]: Refusing update from remote peer 18a35e4031174d4b987facd451db7179: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:03.574219 31313 consensus_queue.cc:1035] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45a6b3554e84a23a79f77d83503f02f" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 43167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:16:03.574383 31319 consensus_queue.cc:1035] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f289d4e00362462eacaa0adad05402a0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:16:03.589445 30488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:03.589596 30488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:03.591831 30488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P deb645eae0be4ceda06bb15762868295: Sending DeleteTablet for 3 replicas of tablet fe29862b4aee4570b57c660f1ea18034
I20260810 07:16:03.592561 30952 tablet_service.cc:1514] Processing DeleteTablet for tablet fe29862b4aee4570b57c660f1ea18034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:51082
I20260810 07:16:03.592691 31169 tablet_service.cc:1514] Processing DeleteTablet for tablet fe29862b4aee4570b57c660f1ea18034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:47768
I20260810 07:16:03.593025 31338 tablet_replica.cc:330] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: stopping tablet replica
I20260810 07:16:03.593086 30725 tablet_service.cc:1514] Processing DeleteTablet for tablet fe29862b4aee4570b57c660f1ea18034 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:58516
I20260810 07:16:03.593364 31338 raft_consensus.cc:2229] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:16:03.593436 31339 tablet_replica.cc:330] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: stopping tablet replica
I20260810 07:16:03.593677 31338 raft_consensus.cc:2258] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:16:03.593675 31339 raft_consensus.cc:2229] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:16:03.593953 31339 raft_consensus.cc:2258] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:16:03.594208 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 30613
I20260810 07:16:03.595139 31338 ts_tablet_manager.cc:1905] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:03.597935 31338 ts_tablet_manager.cc:1918] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:16:03.598021 31338 log.cc:1196] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/fe29862b4aee4570b57c660f1ea18034
I20260810 07:16:03.598317 31338 ts_tablet_manager.cc:1939] T fe29862b4aee4570b57c660f1ea18034 P 18a35e4031174d4b987facd451db7179: Deleting consensus metadata
I20260810 07:16:03.599484 30474 catalog_manager.cc:4841] TS 18a35e4031174d4b987facd451db7179 (127.12.117.130:41401): tablet fe29862b4aee4570b57c660f1ea18034 (table test-table [id=d3ab15d7e88549f78c89a3f64b06e65d]) successfully deleted
I20260810 07:16:03.600462 31339 ts_tablet_manager.cc:1905] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:03.603056 31339 ts_tablet_manager.cc:1918] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:16:03.603163 31339 log.cc:1196] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/fe29862b4aee4570b57c660f1ea18034
I20260810 07:16:03.603494 31339 ts_tablet_manager.cc:1939] T fe29862b4aee4570b57c660f1ea18034 P f289d4e00362462eacaa0adad05402a0: Deleting consensus metadata
I20260810 07:16:03.604599 30472 catalog_manager.cc:4841] TS f289d4e00362462eacaa0adad05402a0 (127.12.117.131:38721): tablet fe29862b4aee4570b57c660f1ea18034 (table test-table [id=d3ab15d7e88549f78c89a3f64b06e65d]) successfully deleted
W20260810 07:16:03.608997 30469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:43167 (error 108)
I20260810 07:16:03.609769 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 30847
I20260810 07:16:03.609951 31295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:03.609683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:43167 (local address 127.0.0.1:58530)
0810 07:16:03.609801 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:03.609873 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:43167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260810 07:16:03.625305 30469 catalog_manager.cc:4584] TS f45a6b3554e84a23a79f77d83503f02f (127.12.117.129:43167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe29862b4aee4570b57c660f1ea18034: Network error: Client connection negotiation failed: client connection to 127.12.117.129:43167: connect: Connection refused (error 111)
I20260810 07:16:03.631429 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 31073
I20260810 07:16:03.640625 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 30446
2026-08-10T07:16:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2505 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[31351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[31351](info): set up 2 sockets
Aug 10 07:16:03 dist-test-slave-gjrz krb5kdc[31351](info): commencing operation
krb5kdc: starting...
W20260810 07:16:04.807402 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:16:04 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:16:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:16:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangeradmin_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangeradmin_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangerlookup_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangerlookup_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
I20260810 07:16:04.963487 12758 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3eVEGZ/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/postgres -l logfile start
2026-08-10 07:16:07.196 UTC [31404] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:16:07.196 UTC [31404] LOG: listening on IPv4 address "127.12.117.148", port 33979
2026-08-10 07:16:07.199 UTC [31404] LOG: listening on Unix socket "/tmp/.s.PGSQL.33979"
2026-08-10 07:16:07.218 UTC [31414] LOG: database system was shut down at 2026-08-10 07:16:06 UTC
2026-08-10 07:16:07.222 UTC [31404] LOG: database system is ready to accept connections
I20260810 07:16:08.308872 12758 mini_postgres.cc:96] Postgres bound to 33979
2026-08-10 07:16:08.316 UTC [31424] FATAL: database "slave" does not exist
127.12.117.148:33979 - accepting connections
I20260810 07:16:08.317515 12758 mini_ranger.cc:161] Starting Ranger...
I20260810 07:16:08.333302 12758 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:16:08.746376 12758 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:16:08.746456 12758 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin
2026-08-10 07:16:09,053 [I] DB FLAVOR :POSTGRES
2026-08-10 07:16:09,053 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:16:09,053 [I] Checking connection..
2026-08-10 07:16:09,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:16:09,300 [I] Checking connection passed.
2026-08-10 07:16:09,300 [I] --------- Verifying version history table ---------
2026-08-10 07:16:09,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:09,539 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:09,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:09,778 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:09,779 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:16:09,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:16:10.000 UTC [31527] WARNING: there is no transaction in progress
2026-08-10 07:16:10,008 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:16:10,009 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:10,265 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:16:10,265 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:16:10,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:16:10,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:16:10,753 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-10 07:16:10,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:11,207 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:16:11,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:11,444 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:16:11,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:11,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:11,925 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:16:11,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:16:12.811 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.826 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.840 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.854 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.868 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.887 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.941 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.947 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.959 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.968 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.977 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.990 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:12.998 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:13.004 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:13.010 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:13.015 UTC [31701] WARNING: there is no transaction in progress
2026-08-10 07:16:13,567 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:16:13,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:16:13,799 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:16:13,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:14,038 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:16:14,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:14,294 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:16:14,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:14,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:14,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-10 07:16:15,006 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:16:15,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:16:15,246 [I] DB_PATCHES have already been applied
I20260810 07:16:15.251504 12758 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3eVEGZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:16:15.251565 12758 mini_ranger.cc:193] Using host: 127.12.117.148
I20260810 07:16:15.255755 12758 mini_ranger.cc:227] Ranger admin URL: http://127.12.117.148:55625
Aug 10, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:16:16 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:16:16 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:16:16.516391 12758 mini_ranger.cc:160] Time spent starting Ranger: real 8.199s user 0.008s sys 0.003s
Aug 10, 2026 7:16:34 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:16:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 7:16:34 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:16:34 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:16:35 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:16:36.096733 12758 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangerkms_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/rangerkms_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.117.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.117.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:16:36 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346196, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:16:36.187436 12758 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:16:36.196991 12758 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:16:36.915405 12758 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:16:36.915490 12758 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms
I20260810 07:16:36.915578 12758 mini_ranger_kms.cc:227] Using postgres at 127.12.117.148:33979
2026-08-10 07:16:37,342 [I] DB FLAVOR :POSTGRES
2026-08-10 07:16:37,342 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:16:37,342 [I] Checking connection
2026-08-10 07:16:37,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:16:37,600 [I] connection success
2026-08-10 07:16:37,600 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:16:37,600 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:16:37,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:16:37,828 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:16:37,828 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:16:37,828 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:16:37,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:33979/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:16:38,088 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:16:38.550948 12758 mini_ranger_kms.cc:310] Created kms service
I20260810 07:16:38.760011 12758 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:16:38.809383 12758 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:16:38 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346196, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:39.179697 12758 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:16:39.179862 12758 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms:/tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:16:39.179888 12758 mini_ranger_kms.cc:240] Using host: 127.12.117.148
I20260810 07:16:39.184214 12758 mini_ranger_kms.cc:276] Ranger KMS PID: 32019
I20260810 07:16:39.184340 12758 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.117.148:60691
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:16:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:16:41 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346201, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:41.760017 12758 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.573s user 0.036s sys 0.274s
I20260810 07:16:41.760146 12758 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:16:41.760254 12758 mini_ranger_kms.cc:401] 127.12.117.148:60691/kms/v1/keys
Aug 10 07:16:41 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346201, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
Aug 10 07:16:42 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346201, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
Aug 10 07:16:44 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:16:44.338176 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45585
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:45585
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:60691/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:44.486779 32100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:44.487016 32100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:44.487052 32100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:44.490985 32100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:16:44.491037 32100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:44.491055 32100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:44.491070 32100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:16:44.491088 32100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:16:44.496371 32100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.117.148:60691/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:45585
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:44.497277 32100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:44.498353 32100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:44.508255 32112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:44.508380 32110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:44.508266 32116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:44.508543 32100 server_base.cc:1034] running on GCE node
I20260810 07:16:44.509130 32100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:44.510143 32100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:44.511322 32100 hybrid_clock.cc:648] HybridClock initialized: now 1786346204511300 us; error 42 us; skew 500 ppm
Aug 10 07:16:44 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346204, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:44.517247 32100 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:16:44.518100 32100 webserver.cc:460] Webserver started at http://127.12.117.190:38157/ using document root <none> and password file <none>
I20260810 07:16:44.518604 32100 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:44.518657 32100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:44.518842 32100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:44 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346204, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:44.584180 32100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "56e4fbd739744c83a1644870334aa552"
format_stamp: "Formatted at 2026-08-10 07:16:44 on dist-test-slave-gjrz"
server_key: "11a9b70a624750894e17e30b9218b786"
server_key_iv: "0101a4c92e6e2167894a91a194435eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:44.584894 32100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "56e4fbd739744c83a1644870334aa552"
format_stamp: "Formatted at 2026-08-10 07:16:44 on dist-test-slave-gjrz"
server_key: "11a9b70a624750894e17e30b9218b786"
server_key_iv: "0101a4c92e6e2167894a91a194435eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:44.588836 32100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 07:16:44.605185 32127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:44.606549 32100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:16:44.606674 32100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "56e4fbd739744c83a1644870334aa552"
format_stamp: "Formatted at 2026-08-10 07:16:44 on dist-test-slave-gjrz"
server_key: "11a9b70a624750894e17e30b9218b786"
server_key_iv: "0101a4c92e6e2167894a91a194435eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:44.606796 32100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:44.616998 32100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:44.619678 32100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:44.619843 32100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:44.628031 32100 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:45585
I20260810 07:16:44.628069 32209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:45585 every 8 connection(s)
I20260810 07:16:44.628938 32100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:16:44.631742 32210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:44.636489 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 32100
I20260810 07:16:44.636610 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:16:44.636938 12758 external_mini_cluster.cc:1428] Setting key 3b839d20486d7aa3643dc921b8329dac
I20260810 07:16:44.637401 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: Bootstrap starting.
I20260810 07:16:44.640146 32210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:44.641350 32210 log.cc:824] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:44.643154 32210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: No bootstrap required, opened a new log
Aug 10 07:16:44 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:44.647778 32210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } }
I20260810 07:16:44.648006 32210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:44.648078 32210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e4fbd739744c83a1644870334aa552, State: Initialized, Role: FOLLOWER
I20260810 07:16:44.648566 32210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } }
I20260810 07:16:44.648726 32210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:16:44.648810 32210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:16:44.648917 32210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:44.649971 32210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } }
I20260810 07:16:44.650334 32210 leader_election.cc:302] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56e4fbd739744c83a1644870334aa552; no voters:
I20260810 07:16:44.650683 32210 leader_election.cc:288] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:16:44.652051 32210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:16:44.654345 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:44.638476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.12.117.190:45585)
0810 07:16:44.638982 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:44.639001 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:44.639042 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:16:44.639834 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:44.639859 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:44.640287 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:44.640671 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:44.641599 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:44.643185 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:44.644476 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:44.645222 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:44.645380 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:44.648322 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:44.648329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:44.648345 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:44.648377 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:44.650307 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:44.650846 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:44.650853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:44.650858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:44.651043 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:44.651412 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:44.651416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:44.651417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:44.651904 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:44.652112 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:44.653054 (+ 942us) server_negotiation.cc:299] Negotiation successful
0810 07:16:44.653351 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":185,"threads_started":1}
I20260810 07:16:44.650874 32218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:44.654850 32218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [term 1 LEADER]: Becoming Leader. State: Replica: 56e4fbd739744c83a1644870334aa552, State: Running, Role: LEADER
I20260810 07:16:44.655160 32218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } }
I20260810 07:16:44.657130 32210 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:16:44.657987 32220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56e4fbd739744c83a1644870334aa552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } } }
I20260810 07:16:44.658107 32220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:44.658421 32238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56e4fbd739744c83a1644870334aa552. Latest consensus state: current_term: 1 leader_uuid: "56e4fbd739744c83a1644870334aa552" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56e4fbd739744c83a1644870334aa552" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45585 } } }
I20260810 07:16:44.658545 32238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552 [sys.catalog]: This master's current role is: LEADER
I20260810 07:16:44.661592 32244 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:16:44.666152 32244 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:16:44.673187 32244 catalog_manager.cc:1347] Generated new cluster ID: abebe7f130214e9085531c7ef0785ec7
I20260810 07:16:44.673245 32244 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:16:44.685596 32244 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:16:44.686810 32244 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:16:44.698109 32244 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: Generated new TSK 0
I20260810 07:16:44.698949 32244 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:16:45.474087 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:60691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:45.623129 32287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:45.623505 32287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:45.623589 32287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:45.628201 32287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:45.628309 32287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:45.628438 32287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:16:45.636123 32287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.117.148:60691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:45.637324 32287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:45.638468 32287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:45.648731 32304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:45.648761 32301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:45.649034 32300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:45.649948 32287 server_base.cc:1034] running on GCE node
I20260810 07:16:45.650344 32287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:45.650960 32287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:45.652132 32287 hybrid_clock.cc:648] HybridClock initialized: now 1786346205652105 us; error 39 us; skew 500 ppm
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:45.655467 32287 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:16:45.656368 32287 webserver.cc:460] Webserver started at http://127.12.117.129:42727/ using document root <none> and password file <none>
I20260810 07:16:45.657047 32287 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:45.657147 32287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:45.657429 32287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:45.676815 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "27e1356383724bc29e86a6c2cfeb59a6"
format_stamp: "Formatted at 2026-08-10 07:16:45 on dist-test-slave-gjrz"
server_key: "bf6211022f91097c402f7497b940c4b1"
server_key_iv: "23f157021d66dde94903c9bd9f1f7a38"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:45.677789 32287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "27e1356383724bc29e86a6c2cfeb59a6"
format_stamp: "Formatted at 2026-08-10 07:16:45 on dist-test-slave-gjrz"
server_key: "bf6211022f91097c402f7497b940c4b1"
server_key_iv: "23f157021d66dde94903c9bd9f1f7a38"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:45.683147 32287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 07:16:45.700212 32315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:45.701713 32287 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260810 07:16:45.701825 32287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "27e1356383724bc29e86a6c2cfeb59a6"
format_stamp: "Formatted at 2026-08-10 07:16:45 on dist-test-slave-gjrz"
server_key: "bf6211022f91097c402f7497b940c4b1"
server_key_iv: "23f157021d66dde94903c9bd9f1f7a38"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:45.701943 32287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:45.724650 32287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:45.727941 32287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:45.728181 32287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:45.728814 32287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:45.729843 32287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:45.729955 32287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:45.730084 32287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:45.730159 32287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:45.743835 32287 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:45669
I20260810 07:16:45.744943 32287 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:16:45.746035 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 32287
I20260810 07:16:45.746145 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:16:45.746454 12758 external_mini_cluster.cc:1428] Setting key 95483b2805bb23566a055ebd936aee9b
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:45.762513 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:45.748657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:52671 (local address 127.12.117.190:45585)
0810 07:16:45.749929 (+ 1272us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:45.749933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:45.749947 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:16:45.749989 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:45.749991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:45.750037 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:45.750124 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:45.751893 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:45.752748 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:45.754812 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:45.755188 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:45.755334 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:45.758366 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:45.758370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:45.758373 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:45.758409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:45.760191 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:45.760773 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:45.760777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:45.760778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:45.760876 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:45.761261 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:45.761265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:45.761267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:45.761492 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:45.761595 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:45.762210 (+ 615us) server_negotiation.cc:299] Negotiation successful
0810 07:16:45.762339 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1184,"thread_start_us":55,"threads_started":1}
I20260810 07:16:45.763435 32503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:45.747628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45585 (local address 127.12.117.129:52671)
0810 07:16:45.748321 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:45.748376 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:45.749530 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:45.750350 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:45.750359 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:45.750854 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:45.751707 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:45.751730 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:45.752883 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:45.752887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:45.754654 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:45.754666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:45.755386 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:45.755391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:45.755649 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:45.756315 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:45.756335 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:45.758179 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:45.760312 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:45.760317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:45.760332 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:45.760670 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:45.760971 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:45.760974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:45.760976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:45.761144 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:45.761606 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:45.761612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:45.761916 (+ 304us) client_negotiation.cc:769] Sending connection context
0810 07:16:45.762124 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 07:16:45.762455 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":120,"threads_started":1}
I20260810 07:16:45.764961 32495 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45585
I20260810 07:16:45.765179 32495 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:45.765600 32495 heartbeater.cc:511] Master 127.12.117.190:45585 requested a full tablet report, sending...
I20260810 07:16:45.767190 32146 ts_manager.cc:194] Registered new tserver with Master: 27e1356383724bc29e86a6c2cfeb59a6 (127.12.117.129:45669)
I20260810 07:16:45.768728 32146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:52671
I20260810 07:16:45.768967 32487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:45669 every 8 connection(s)
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:16:45.792830 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:60691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 07:16:45.960677 32512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:45.961010 32512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:45.961143 32512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:45.966902 32512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:45.966974 32512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:45.967057 32512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:16:45.974515 32512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.117.148:60691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:45.975818 32512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:45.977171 32512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:45.989516 32530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:45.989562 32526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:45.990101 32523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:45.990871 32512 server_base.cc:1034] running on GCE node
I20260810 07:16:45.991457 32512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:45.992164 32512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:45.995445 32512 hybrid_clock.cc:648] HybridClock initialized: now 1786346205995415 us; error 37 us; skew 500 ppm
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:45.999100 32512 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:16:46.000197 32512 webserver.cc:460] Webserver started at http://127.12.117.130:46489/ using document root <none> and password file <none>
I20260810 07:16:46.000949 32512 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:46.001083 32512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:46.001384 32512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:45 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:46.019559 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "c90b55d5133844f580c74605dfadc6fb"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "f549934f666eef44dfbac41c218c16d3"
server_key_iv: "167fcef9ab0d980d82ddd79583ee5eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.020509 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "c90b55d5133844f580c74605dfadc6fb"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "f549934f666eef44dfbac41c218c16d3"
server_key_iv: "167fcef9ab0d980d82ddd79583ee5eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.028800 32512 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260810 07:16:46.049868 32539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.051626 32512 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260810 07:16:46.051753 32512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "c90b55d5133844f580c74605dfadc6fb"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "f549934f666eef44dfbac41c218c16d3"
server_key_iv: "167fcef9ab0d980d82ddd79583ee5eb4"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.051882 32512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:46.077533 32512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:46.081322 32512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:46.081555 32512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:46.082289 32512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:46.083391 32512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:46.083494 32512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.083588 32512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:46.083648 32512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.099462 32512 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:42147
I20260810 07:16:46.100690 32512 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:16:46.106565 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 32512
I20260810 07:16:46.106690 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:16:46.107004 12758 external_mini_cluster.cc:1428] Setting key df63b9654c44c56ef590ee360ba63cf9
I20260810 07:16:46.111001 32724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:42147 every 8 connection(s)
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:46.118760 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.103643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:53059 (local address 127.12.117.190:45585)
0810 07:16:46.103981 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.103986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.105937 (+ 1951us) server_negotiation.cc:407] Connection header received
0810 07:16:46.107247 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.107251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.107326 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.107383 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:46.109768 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.110607 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.112641 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.112882 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.112984 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.115536 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:46.115540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:46.115543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:46.115567 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:46.116888 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.117347 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.117351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.117353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.117427 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.117731 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.117735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.117737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.117908 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:46.118001 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.118458 (+ 457us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.118592 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260810 07:16:46.119580 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.104164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45585 (local address 127.12.117.130:53059)
0810 07:16:46.105019 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.105802 (+ 783us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.106967 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.108472 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.108481 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.108945 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:46.109580 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.109600 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.111390 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.111392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.112409 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.112420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.113023 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.113027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.113243 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.113831 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:46.113851 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:46.115389 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:46.116996 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:46.117002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:46.117016 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:46.117256 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:46.117512 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:46.117515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:46.117517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:46.117648 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:46.118004 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:46.118010 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:46.118243 (+ 233us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.118402 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.118695 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":176,"threads_started":1}
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:16:46.120783 32725 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45585
I20260810 07:16:46.121009 32725 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:46.121526 32725 heartbeater.cc:511] Master 127.12.117.190:45585 requested a full tablet report, sending...
I20260810 07:16:46.122604 32146 ts_manager.cc:194] Registered new tserver with Master: c90b55d5133844f580c74605dfadc6fb (127.12.117.130:42147)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:16:46.123368 32146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:53059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:16:46.140691 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:60691/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:41293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:16:46.282594 32730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:16:46.282835 32730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:16:46.282899 32730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:16:46.286844 32730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:16:46.286900 32730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:16:46.286978 32730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:16:46.292990 32730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.117.148:60691/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:45585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:16:46.294320 32730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:16:46.295475 32730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:16:46.312155 32741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:46.312592 32742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:16:46.313508 32744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:16:46.313793 32730 server_base.cc:1034] running on GCE node
I20260810 07:16:46.314397 32730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:16:46.315253 32730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:16:46.316501 32730 hybrid_clock.cc:648] HybridClock initialized: now 1786346206316438 us; error 76 us; skew 500 ppm
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346206, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:16:46.319972 32730 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:16:46.321039 32730 webserver.cc:460] Webserver started at http://127.12.117.131:41313/ using document root <none> and password file <none>
I20260810 07:16:46.321763 32730 fs_manager.cc:359] Metadata directory not provided
I20260810 07:16:46.321841 32730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:16:46.322209 32730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346206, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:46.357818 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "0b019bcb644849aeabd4084b412ab023"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "9a8838def0f26340e0fd6c95155e93dc"
server_key_iv: "595d674539232a64d4ea64b6680bee75"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.360420 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "0b019bcb644849aeabd4084b412ab023"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "9a8838def0f26340e0fd6c95155e93dc"
server_key_iv: "595d674539232a64d4ea64b6680bee75"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.367225 32730 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 07:16:46.391492 32761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.393139 32730 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260810 07:16:46.393328 32730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "0b019bcb644849aeabd4084b412ab023"
format_stamp: "Formatted at 2026-08-10 07:16:46 on dist-test-slave-gjrz"
server_key: "9a8838def0f26340e0fd6c95155e93dc"
server_key_iv: "595d674539232a64d4ea64b6680bee75"
server_key_version: "kuduclusterkey@0"
I20260810 07:16:46.393524 32730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:16:46.411723 32730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:16:46.415009 32730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:16:46.415189 32730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:16:46.415863 32730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:16:46.416862 32730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:16:46.416936 32730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.417047 32730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:16:46.417088 32730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:16:46.430392 32730 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:38905
I20260810 07:16:46.430531 468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:38905 every 8 connection(s)
I20260810 07:16:46.431595 32730 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:16:46.439075 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 32730
I20260810 07:16:46.439205 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:16:46.439522 12758 external_mini_cluster.cc:1428] Setting key b0a212f4dad8496acad746bf3f74b9f6
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346206, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:16:46.448892 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.433971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:42225 (local address 127.12.117.190:45585)
0810 07:16:46.434101 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.434105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.435230 (+ 1125us) server_negotiation.cc:407] Connection header received
0810 07:16:46.436451 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.436455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.436509 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.436600 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:46.437986 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.438749 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.441602 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.441964 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.442035 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.445079 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:46.445083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:46.445086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:46.445112 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:46.446657 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.447225 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.447229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.447231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.447363 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.447747 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.447751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.447753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.447975 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:46.448078 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.448624 (+ 546us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.448743 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:16:46.449765 472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.434437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:45585 (local address 127.12.117.131:42225)
0810 07:16:46.435076 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.435131 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.436206 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.436783 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.436792 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.437118 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:16:46.437813 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.437833 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.439924 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.439928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.441463 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.441475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.442216 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.442219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.442400 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.443028 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:16:46.443047 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:16:46.444898 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:16:46.446772 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:46.446778 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:46.446792 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:46.447101 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:46.447459 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:16:46.447462 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:16:46.447464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:16:46.447631 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:16:46.448074 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:16:46.448079 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:16:46.448348 (+ 269us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.448535 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.448822 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":153,"threads_started":1}
I20260810 07:16:46.450917 469 heartbeater.cc:347] Connected to a master server at 127.12.117.190:45585
I20260810 07:16:46.451169 469 heartbeater.cc:464] Registering TS with master...
I20260810 07:16:46.451821 469 heartbeater.cc:511] Master 127.12.117.190:45585 requested a full tablet report, sending...
I20260810 07:16:46.453071 32153 ts_manager.cc:194] Registered new tserver with Master: 0b019bcb644849aeabd4084b412ab023 (127.12.117.131:38905)
I20260810 07:16:46.453651 32153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:42225
I20260810 07:16:46.464718 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:16:46.476001 32506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.466790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47628 (local address 127.12.117.190:45585)
0810 07:16:46.466892 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.466895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.467136 (+ 241us) server_negotiation.cc:407] Connection header received
0810 07:16:46.467247 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.467251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.467335 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.467398 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:16:46.468079 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.468576 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.469384 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.469602 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.469688 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.471020 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:16:46.471023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:16:46.471026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:16:46.471052 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:16:46.473845 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.474562 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.474566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.474568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.474665 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:16:46.474999 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:16:46.475003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:16:46.475005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:16:46.475219 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:16:46.475355 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.475709 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.475824 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:16:46.479511 32153 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:16:46.483493 32153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:16:46.500854 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.493819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:42147 (local address 127.0.0.1:44484)
0810 07:16:46.494771 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.494818 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.495037 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.495827 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.495833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.495856 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:46.495986 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.495992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.497460 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.497463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.498956 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.498968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.500308 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.500311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.500419 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.500472 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.500592 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.500713 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":107,"threads_started":1}
I20260810 07:16:46.500972 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.494012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44484 (local address 127.12.117.130:42147)
0810 07:16:46.494489 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.494505 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.494921 (+ 416us) server_negotiation.cc:407] Connection header received
0810 07:16:46.495230 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.495237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.495477 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.495621 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:46.496120 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.497300 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.499131 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.499972 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.500068 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.500205 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.500711 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.500817 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":106,"threads_started":1}
I20260810 07:16:46.504415 32621 tablet_service.cc:1467] Processing CreateTablet for tablet f58393b37b704736867822c883ffef28 (DEFAULT_TABLE table=test-table [id=76db7f8399fe49368f9caf75268a673b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:46.505542 32621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58393b37b704736867822c883ffef28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:46.507170 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.496665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38905 (local address 127.0.0.1:46688)
0810 07:16:46.497423 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.497434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.497504 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.502797 (+ 5293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.502799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.502816 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:46.502892 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.502897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.504341 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.504344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.505399 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.505409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.506790 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.506794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.506879 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.506902 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.506970 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.507045 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":85,"threads_started":1}
I20260810 07:16:46.507573 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.497087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.12.117.131:38905)
0810 07:16:46.502199 (+ 5112us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.502211 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.502235 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:16:46.502306 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.502311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.502511 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.502658 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:46.503019 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.504202 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.505651 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.506496 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.506610 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.506725 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.507337 (+ 612us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.507433 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4303,"thread_start_us":196,"threads_started":1}
I20260810 07:16:46.509337 502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.494873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45669 (local address 127.0.0.1:55324)
0810 07:16:46.497166 (+ 2293us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.497180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.497262 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.503795 (+ 6533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.503798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.503814 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:46.503909 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.503931 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.505417 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.505420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.506661 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.506673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.508979 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.508982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.509074 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.509094 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.509157 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.509218 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2235,"thread_start_us":102,"threads_started":1}
I20260810 07:16:46.509800 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.495294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.12.117.129:45669)
0810 07:16:46.503456 (+ 8162us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.503461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.503480 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:16:46.503538 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.503542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.503672 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.503801 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:46.504058 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.505267 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.506796 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.508671 (+ 1875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.508764 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.508872 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.509576 (+ 704us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.509672 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8053,"thread_start_us":94,"threads_started":1}
I20260810 07:16:46.510279 364 tablet_service.cc:1467] Processing CreateTablet for tablet f58393b37b704736867822c883ffef28 (DEFAULT_TABLE table=test-table [id=76db7f8399fe49368f9caf75268a673b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:46.511643 364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58393b37b704736867822c883ffef28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:46.512655 32391 tablet_service.cc:1467] Processing CreateTablet for tablet f58393b37b704736867822c883ffef28 (DEFAULT_TABLE table=test-table [id=76db7f8399fe49368f9caf75268a673b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:16:46.513620 32391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58393b37b704736867822c883ffef28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:16:46.525269 508 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Bootstrap starting.
I20260810 07:16:46.527595 506 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Bootstrap starting.
I20260810 07:16:46.527654 507 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: Bootstrap starting.
I20260810 07:16:46.528574 508 tablet_bootstrap.cc:654] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:46.529597 508 log.cc:824] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:46.530228 507 tablet_bootstrap.cc:654] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:46.530339 506 tablet_bootstrap.cc:654] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Neither blocks nor log segments found. Creating new log.
I20260810 07:16:46.531315 507 log.cc:824] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:46.532557 508 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: No bootstrap required, opened a new log
I20260810 07:16:46.532685 506 log.cc:824] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:16:46.532745 508 ts_tablet_manager.cc:1397] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 07:16:46.534813 507 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: No bootstrap required, opened a new log
I20260810 07:16:46.534999 507 ts_tablet_manager.cc:1397] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260810 07:16:46.536718 508 raft_consensus.cc:348] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.537017 508 raft_consensus.cc:374] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:46.537081 508 raft_consensus.cc:729] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b019bcb644849aeabd4084b412ab023, State: Initialized, Role: FOLLOWER
I20260810 07:16:46.537725 508 consensus_queue.cc:260] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.538932 508 ts_tablet_manager.cc:1428] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:16:46.539389 469 heartbeater.cc:503] Master 127.12.117.190:45585 was elected leader, sending a full tablet report...
I20260810 07:16:46.540266 507 raft_consensus.cc:348] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.540601 507 raft_consensus.cc:374] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:46.541932 506 tablet_bootstrap.cc:492] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: No bootstrap required, opened a new log
I20260810 07:16:46.542124 506 ts_tablet_manager.cc:1397] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260810 07:16:46.546164 507 raft_consensus.cc:729] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27e1356383724bc29e86a6c2cfeb59a6, State: Initialized, Role: FOLLOWER
I20260810 07:16:46.546690 507 consensus_queue.cc:260] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.547042 506 raft_consensus.cc:348] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.547377 506 raft_consensus.cc:374] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:16:46.547477 506 raft_consensus.cc:729] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90b55d5133844f580c74605dfadc6fb, State: Initialized, Role: FOLLOWER
I20260810 07:16:46.547875 507 ts_tablet_manager.cc:1428] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260810 07:16:46.548038 506 consensus_queue.cc:260] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.548239 32495 heartbeater.cc:503] Master 127.12.117.190:45585 was elected leader, sending a full tablet report...
I20260810 07:16:46.548740 32725 heartbeater.cc:503] Master 127.12.117.190:45585 was elected leader, sending a full tablet report...
I20260810 07:16:46.549038 506 ts_tablet_manager.cc:1428] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
W20260810 07:16:46.603873 32726 tablet.cc:2369] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:16:46.611322 519 raft_consensus.cc:482] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:16:46.611517 519 raft_consensus.cc:504] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.612896 519 leader_election.cc:288] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27e1356383724bc29e86a6c2cfeb59a6 (127.12.117.129:45669), 0b019bcb644849aeabd4084b412ab023 (127.12.117.131:38905)
I20260810 07:16:46.618034 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.613492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38905 (local address 127.12.117.130:37147)
0810 07:16:46.613765 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.613780 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.613872 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.614503 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.614507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.614530 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:46.614651 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.614658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.615725 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.615729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.616724 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.616734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.617628 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.617631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.617730 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.617754 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.617823 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.617905 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260810 07:16:46.618388 505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.614036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:37147 (local address 127.12.117.131:38905)
0810 07:16:46.614161 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.614166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.614179 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:16:46.614237 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.614240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.614291 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.614362 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:46.614782 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.615553 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.616849 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.617383 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.617500 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.617561 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.618198 (+ 637us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.618273 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:16:46.619596 388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58393b37b704736867822c883ffef28" candidate_uuid: "c90b55d5133844f580c74605dfadc6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b019bcb644849aeabd4084b412ab023" is_pre_election: true
I20260810 07:16:46.619925 388 raft_consensus.cc:2454] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90b55d5133844f580c74605dfadc6fb in term 0.
I20260810 07:16:46.620477 32548 leader_election.cc:302] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b019bcb644849aeabd4084b412ab023, c90b55d5133844f580c74605dfadc6fb; no voters:
I20260810 07:16:46.620785 519 raft_consensus.cc:2781] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:16:46.620879 519 raft_consensus.cc:482] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:16:46.620916 519 raft_consensus.cc:3037] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:46.622555 519 raft_consensus.cc:504] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.622707 501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.616104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:42977 (local address 127.12.117.129:45669)
0810 07:16:46.616234 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.616238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.617121 (+ 883us) server_negotiation.cc:407] Connection header received
0810 07:16:46.617313 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.617317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.617369 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.617446 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:16:46.618091 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.618764 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.621360 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.621977 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.622078 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.622132 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.622512 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.622593 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 07:16:46.622916 519 leader_election.cc:288] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [CANDIDATE]: Term 1 election: Requested vote from peers 27e1356383724bc29e86a6c2cfeb59a6 (127.12.117.129:45669), 0b019bcb644849aeabd4084b412ab023 (127.12.117.131:38905)
I20260810 07:16:46.623234 388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58393b37b704736867822c883ffef28" candidate_uuid: "c90b55d5133844f580c74605dfadc6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b019bcb644849aeabd4084b412ab023"
I20260810 07:16:46.623380 388 raft_consensus.cc:3037] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:46.624392 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.615625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45669 (local address 127.12.117.130:42977)
0810 07:16:46.617013 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.617026 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:16:46.617188 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:16:46.617720 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:16:46.617722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:16:46.617741 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:16:46.617953 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.617961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.619402 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.619405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.621222 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:16:46.621232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.622258 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.622261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:16:46.622351 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.622387 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 07:16:46.622875 (+ 488us) client_negotiation.cc:240] Negotiation successful
0810 07:16:46.624213 (+ 1338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":12,"thread_start_us":136,"threads_started":1}
I20260810 07:16:46.625075 388 raft_consensus.cc:2454] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90b55d5133844f580c74605dfadc6fb in term 1.
I20260810 07:16:46.625522 32416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58393b37b704736867822c883ffef28" candidate_uuid: "c90b55d5133844f580c74605dfadc6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e1356383724bc29e86a6c2cfeb59a6" is_pre_election: true
I20260810 07:16:46.625797 32416 raft_consensus.cc:2454] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90b55d5133844f580c74605dfadc6fb in term 0.
I20260810 07:16:46.626109 32412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58393b37b704736867822c883ffef28" candidate_uuid: "c90b55d5133844f580c74605dfadc6fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27e1356383724bc29e86a6c2cfeb59a6"
I20260810 07:16:46.626216 32412 raft_consensus.cc:3037] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:16:46.627477 32412 raft_consensus.cc:2454] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90b55d5133844f580c74605dfadc6fb in term 1.
I20260810 07:16:46.627828 32545 leader_election.cc:302] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27e1356383724bc29e86a6c2cfeb59a6, c90b55d5133844f580c74605dfadc6fb; no voters:
I20260810 07:16:46.628001 519 raft_consensus.cc:2781] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:16:46.628187 519 raft_consensus.cc:686] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 1 LEADER]: Becoming Leader. State: Replica: c90b55d5133844f580c74605dfadc6fb, State: Running, Role: LEADER
I20260810 07:16:46.628501 519 consensus_queue.cc:237] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } }
I20260810 07:16:46.632130 32146 catalog_manager.cc:5495] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb reported cstate change: term changed from 0 to 1, leader changed from <none> to c90b55d5133844f580c74605dfadc6fb (127.12.117.130). New cstate: current_term: 1 leader_uuid: "c90b55d5133844f580c74605dfadc6fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c90b55d5133844f580c74605dfadc6fb" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 } health_report { overall_health: UNKNOWN } } }
Aug 10 07:16:46 dist-test-slave-gjrz krb5kdc[31351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346205, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:16:46.664554 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.657038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44494 (local address 127.12.117.130:42147)
0810 07:16:46.657152 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0810 07:16:46.657156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:16:46.657169 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:16:46.657244 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:16:46.657247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:16:46.657306 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:16:46.657404 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:16:46.657806 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.658646 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.662135 (+ 3489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:16:46.662483 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:16:46.662582 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:16:46.663050 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:16:46.663610 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:16:46.663730 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:16:46.664358 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 07:16:46.664417 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:16:46.671876 388 raft_consensus.cc:1261] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 1 FOLLOWER]: Refusing update from remote peer c90b55d5133844f580c74605dfadc6fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:46.672660 519 consensus_queue.cc:1035] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b019bcb644849aeabd4084b412ab023" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:16:46.679701 32412 raft_consensus.cc:1261] T f58393b37b704736867822c883ffef28 P 27e1356383724bc29e86a6c2cfeb59a6 [term 1 FOLLOWER]: Refusing update from remote peer c90b55d5133844f580c74605dfadc6fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:16:46.680348 519 consensus_queue.cc:1035] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "27e1356383724bc29e86a6c2cfeb59a6" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:16:46.688005 470 tablet.cc:2369] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:16:46.692022 527 mvcc.cc:205] Tried to move back new op lower bound from 7316874062519296000 to 7316874062352789504. Current Snapshot: MvccSnapshot[applied={T|T < 7316874062519296000}]
I20260810 07:16:46.695887 534 mvcc.cc:205] Tried to move back new op lower bound from 7316874062519296000 to 7316874062352789504. Current Snapshot: MvccSnapshot[applied={T|T < 7316874062519296000}]
I20260810 07:16:46.696339 531 mvcc.cc:205] Tried to move back new op lower bound from 7316874062519296000 to 7316874062352789504. Current Snapshot: MvccSnapshot[applied={T|T < 7316874062519296000}]
I20260810 07:16:46.705633 32146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:46.705781 32146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:16:46.708549 32146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56e4fbd739744c83a1644870334aa552: Sending DeleteTablet for 3 replicas of tablet f58393b37b704736867822c883ffef28
I20260810 07:16:46.709398 32391 tablet_service.cc:1514] Processing DeleteTablet for tablet f58393b37b704736867822c883ffef28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:46 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55324
I20260810 07:16:46.709510 364 tablet_service.cc:1514] Processing DeleteTablet for tablet f58393b37b704736867822c883ffef28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:46 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:46688
I20260810 07:16:46.710191 32621 tablet_service.cc:1514] Processing DeleteTablet for tablet f58393b37b704736867822c883ffef28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:16:46 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:44484
I20260810 07:16:46.711438 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 32287
I20260810 07:16:46.719054 548 tablet_replica.cc:330] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: stopping tablet replica
I20260810 07:16:46.724982 548 raft_consensus.cc:2229] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:16:46.725467 548 raft_consensus.cc:2258] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:16:46.727351 548 ts_tablet_manager.cc:1905] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:46.730443 548 ts_tablet_manager.cc:1918] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:16:46.730608 548 log.cc:1196] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/f58393b37b704736867822c883ffef28
I20260810 07:16:46.731029 548 ts_tablet_manager.cc:1939] T f58393b37b704736867822c883ffef28 P 0b019bcb644849aeabd4084b412ab023: Deleting consensus metadata
I20260810 07:16:46.731504 549 tablet_replica.cc:330] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: stopping tablet replica
I20260810 07:16:46.731873 549 raft_consensus.cc:2229] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:16:46.735337 549 raft_consensus.cc:2258] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 07:16:46.740157 32129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:45669 (error 108)
I20260810 07:16:46.742225 549 ts_tablet_manager.cc:1905] T f58393b37b704736867822c883ffef28 P c90b55d5133844f580c74605dfadc6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:16:46.747447 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 32512
I20260810 07:16:46.753801 32132 catalog_manager.cc:4841] TS 0b019bcb644849aeabd4084b412ab023 (127.12.117.131:38905): tablet f58393b37b704736867822c883ffef28 (table test-table [id=76db7f8399fe49368f9caf75268a673b]) successfully deleted
I20260810 07:16:46.762260 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 32730
I20260810 07:16:46.776309 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.775736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45669 (local address 127.0.0.1:55332)
0810 07:16:46.776127 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.776214 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289}
I20260810 07:16:46.776309 502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:16:46.775984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:42147 (local address 127.0.0.1:44504)
0810 07:16:46.776148 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:16:46.776204 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.130:42147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260810 07:16:46.779812 32133 catalog_manager.cc:4584] TS c90b55d5133844f580c74605dfadc6fb (127.12.117.130:42147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f58393b37b704736867822c883ffef28: Network error: Client connection negotiation failed: client connection to 127.12.117.130:42147: connect: Connection refused (error 111)
I20260810 07:16:46.781289 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 32100
I20260810 07:16:46.798136 12758 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:16:47.141513 12758 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:16:47.141637 12758 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:16:47.485459 12758 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:16:47.485 UTC [31404] LOG: received smart shutdown request
2026-08-10 07:16:47.489 UTC [31404] LOG: background worker "logical replication launcher" (PID 31422) exited with exit code 1
2026-08-10 07:16:47.489 UTC [31415] LOG: shutting down
2026-08-10 07:16:47.620 UTC [31404] LOG: database system is shut down
2026-08-10T07:16:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44073 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:16:47 dist-test-slave-gjrz krb5kdc[567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:16:47 dist-test-slave-gjrz krb5kdc[567](info): set up 2 sockets
Aug 10 07:16:47 dist-test-slave-gjrz krb5kdc[567](info): commencing operation
krb5kdc: starting...
W20260810 07:16:48.883067 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:16:48 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:16:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangeradmin_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangeradmin_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangerlookup_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangerlookup_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
I20260810 07:16:49.035885 12758 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task3eVEGZ/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/postgres -l logfile start
2026-08-10 07:16:51.204 UTC [615] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 07:16:51.204 UTC [615] LOG: listening on IPv4 address "127.12.117.148", port 43665
2026-08-10 07:16:51.207 UTC [615] LOG: listening on Unix socket "/tmp/.s.PGSQL.43665"
2026-08-10 07:16:51.234 UTC [619] LOG: database system was shut down at 2026-08-10 07:16:50 UTC
2026-08-10 07:16:51.238 UTC [615] LOG: database system is ready to accept connections
I20260810 07:16:52.303869 12758 mini_postgres.cc:96] Postgres bound to 43665
2026-08-10 07:16:52.310 UTC [630] FATAL: database "slave" does not exist
127.12.117.148:43665 - accepting connections
I20260810 07:16:52.311348 12758 mini_ranger.cc:161] Starting Ranger...
I20260810 07:16:52.325322 12758 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 07:16:52.709321 12758 mini_ranger.cc:87] Created ranger Postgres database
I20260810 07:16:52.709388 12758 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin
2026-08-10 07:16:52,990 [I] DB FLAVOR :POSTGRES
2026-08-10 07:16:52,990 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:16:52,990 [I] Checking connection..
2026-08-10 07:16:52,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 07:16:53,247 [I] Checking connection passed.
2026-08-10 07:16:53,248 [I] --------- Verifying version history table ---------
2026-08-10 07:16:53,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:53,500 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:53,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:53,754 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 07:16:53,754 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 07:16:53,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 07:16:54.000 UTC [746] WARNING: there is no transaction in progress
2026-08-10 07:16:54,006 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 07:16:54,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 07:16:54,248 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 07:16:54,248 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:16:54,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 07:16:54,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 07:16:54,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjrz.c.gcp-upstream.internal','N') ;"
2026-08-10 07:16:54,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:55,175 [I] Table x_portal_user does not exist in database ranger
2026-08-10 07:16:55,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:55,400 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 07:16:55,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:55,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:55,880 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 07:16:55,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 07:16:56.721 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.735 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.749 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.761 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.775 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.790 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.839 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.844 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.857 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.865 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.874 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.887 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.895 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.900 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.905 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:56.910 UTC [934] WARNING: there is no transaction in progress
2026-08-10 07:16:57,447 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 07:16:57,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 07:16:57,692 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 07:16:57,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 07:16:57,919 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 07:16:57,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 07:16:58,151 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 07:16:58,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 07:16:58,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 07:16:58,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjrz.c.gcp-upstream.internal';"
2026-08-10 07:16:58,864 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 07:16:58,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 07:16:59,116 [I] DB_PATCHES have already been applied
I20260810 07:16:59.121565 12758 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task3eVEGZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 07:16:59.121628 12758 mini_ranger.cc:193] Using host: 127.12.117.148
I20260810 07:16:59.125692 12758 mini_ranger.cc:227] Ranger admin URL: http://127.12.117.148:41999
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 7:16:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 07:17:00.378985 12758 mini_ranger.cc:160] Time spent starting Ranger: real 8.068s user 0.002s sys 0.007s
Aug 10, 2026 7:17:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 7:17:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 7:17:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 7:17:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 7:17:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 07:17:20.455492 12758 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.117.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.117.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangerkms_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.117.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/rangerkms_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.117.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.117.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.117.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/HTTP_127.12.117.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 07:17:20 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346240, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 07:17:20.548707 12758 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 07:17:20.558444 12758 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 07:17:21.295251 12758 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 07:17:21.295369 12758 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms
I20260810 07:17:21.295388 12758 mini_ranger_kms.cc:227] Using postgres at 127.12.117.148:43665
2026-08-10 07:17:21,452 [I] DB FLAVOR :POSTGRES
2026-08-10 07:17:21,452 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 07:17:21,452 [I] Checking connection
2026-08-10 07:17:21,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 07:17:21,722 [I] connection success
2026-08-10 07:17:21,722 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 07:17:21,722 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 07:17:21,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 07:17:21,988 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 07:17:21,988 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 07:17:21,988 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 07:17:21,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.117.148:43665/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 07:17:22,298 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 07:17:22.727662 12758 mini_ranger_kms.cc:310] Created kms service
I20260810 07:17:22.900444 12758 mini_ranger_kms.cc:326] Created kudu user
I20260810 07:17:22.942226 12758 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 07:17:22 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346240, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:23.332913 12758 mini_ranger_kms.cc:384] Added ranger policy
I20260810 07:17:23.333060 12758 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms:/tmp/dist-test-task3eVEGZ/build/debug/bin/postgresql.jar:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task3eVEGZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 07:17:23.333081 12758 mini_ranger_kms.cc:240] Using host: 127.12.117.148
I20260810 07:17:23.338024 12758 mini_ranger_kms.cc:276] Ranger KMS PID: 1237
I20260810 07:17:23.338090 12758 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.117.148:44797
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 7:17:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 07:17:25 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 07:17:25 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:25.880330 12758 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.332s user 0.004s sys 0.137s
I20260810 07:17:25.880430 12758 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 07:17:25.880481 12758 mini_ranger_kms.cc:401] 127.12.117.148:44797/kms/v1/keys
Aug 10 07:17:26 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346245, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.117.148@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
Aug 10 07:17:28 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:17:28.180166 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:35159
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:35159
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:28.287905 1341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:28.288144 1341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:28.288182 1341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:28.292222 1341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:28.292274 1341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:28.292292 1341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:28.292306 1341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:28.292320 1341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:28.297852 1341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.117.148:44797/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:35159
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:28.298851 1341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:28.299849 1341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:28.309401 1351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:28.309547 1341 server_base.cc:1034] running on GCE node
W20260810 07:17:28.309641 1354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:28.309471 1349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:28.310166 1341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:28.311205 1341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:28.312381 1341 hybrid_clock.cc:648] HybridClock initialized: now 1786346248312352 us; error 48 us; skew 500 ppm
Aug 10 07:17:28 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346248, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:28.317981 1341 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:28.318856 1341 webserver.cc:460] Webserver started at http://127.12.117.190:41751/ using document root <none> and password file <none>
I20260810 07:17:28.319384 1341 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:28.319439 1341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:28.319631 1341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:28 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346248, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:28.382074 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "a79767011e7146d19ecd4c011df6983f"
format_stamp: "Formatted at 2026-08-10 07:17:28 on dist-test-slave-gjrz"
server_key: "aeac024ecdf09e242f8e0ed9e799ceba"
server_key_iv: "8d3350f2851751edbfc00c9912dfa696"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:28.382730 1341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "a79767011e7146d19ecd4c011df6983f"
format_stamp: "Formatted at 2026-08-10 07:17:28 on dist-test-slave-gjrz"
server_key: "aeac024ecdf09e242f8e0ed9e799ceba"
server_key_iv: "8d3350f2851751edbfc00c9912dfa696"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:28.386634 1341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:17:28.403568 1366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:28.405200 1341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:28.405334 1341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "a79767011e7146d19ecd4c011df6983f"
format_stamp: "Formatted at 2026-08-10 07:17:28 on dist-test-slave-gjrz"
server_key: "aeac024ecdf09e242f8e0ed9e799ceba"
server_key_iv: "8d3350f2851751edbfc00c9912dfa696"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:28.405459 1341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:28.425130 1341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:28.427646 1341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:28.427839 1341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:28.434702 1341 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:35159
I20260810 07:17:28.434713 1458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:35159 every 8 connection(s)
I20260810 07:17:28.435659 1341 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:28.436892 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 1341
I20260810 07:17:28.437003 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:17:28.437327 12758 external_mini_cluster.cc:1428] Setting key 84862864e7dab40e05a424f3cdb3e490
I20260810 07:17:28.438796 1460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:28.444895 1460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Bootstrap starting.
Aug 10 07:17:28 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:28.447047 1460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:28.447818 1460 log.cc:824] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:28.449513 1460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: No bootstrap required, opened a new log
I20260810 07:17:28.450933 1464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:28.438618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.12.117.190:35159)
0810 07:17:28.438999 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:28.439014 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:28.439050 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:17:28.439740 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:28.439760 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:28.440085 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:28.440350 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:28.441169 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:28.442141 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:28.443292 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:28.443739 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:28.443849 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:28.446411 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:28.446417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:28.446430 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:28.446456 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:28.448425 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:28.448828 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:28.448833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:28.448838 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:28.448940 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:28.449209 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:28.449212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:28.449214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:28.449589 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:28.449767 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:28.450052 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 07:17:28.450299 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":104,"threads_started":1}
I20260810 07:17:28.452571 1460 raft_consensus.cc:348] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:28.452818 1460 raft_consensus.cc:374] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:28.452880 1460 raft_consensus.cc:729] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79767011e7146d19ecd4c011df6983f, State: Initialized, Role: FOLLOWER
I20260810 07:17:28.453264 1460 consensus_queue.cc:260] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:28.453388 1460 raft_consensus.cc:388] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:28.453449 1460 raft_consensus.cc:482] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:28.453541 1460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:28.454329 1460 raft_consensus.cc:504] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:28.454625 1460 leader_election.cc:302] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a79767011e7146d19ecd4c011df6983f; no voters:
I20260810 07:17:28.454875 1460 leader_election.cc:288] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:28.454968 1467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:28.455477 1467 raft_consensus.cc:686] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 LEADER]: Becoming Leader. State: Replica: a79767011e7146d19ecd4c011df6983f, State: Running, Role: LEADER
I20260810 07:17:28.455789 1467 consensus_queue.cc:237] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:28.455952 1460 sys_catalog.cc:564] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:28.457418 1470 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a79767011e7146d19ecd4c011df6983f. Latest consensus state: current_term: 1 leader_uuid: "a79767011e7146d19ecd4c011df6983f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } } }
I20260810 07:17:28.457415 1468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a79767011e7146d19ecd4c011df6983f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } } }
I20260810 07:17:28.457574 1470 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:28.457574 1468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:28.457929 1493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:28.458244 1460 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:17:28.463824 1493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:28.470392 1493 catalog_manager.cc:1347] Generated new cluster ID: a4d3e028dc9b4349be68e1dc343db2f0
I20260810 07:17:28.470458 1493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:28.481755 1493 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:28.482901 1493 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:28.494904 1493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Generated new TSK 0
I20260810 07:17:28.495553 1493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:17:29.176806 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:29.292044 1547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:29.292295 1547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:29.292353 1547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:29.297451 1547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:29.297508 1547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:29.297567 1547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:29.302762 1547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:29.303875 1547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:29.304900 1547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:29.315464 1561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:29.315568 1559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:29.315990 1558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:29.316123 1547 server_base.cc:1034] running on GCE node
I20260810 07:17:29.317148 1547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:29.317891 1547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:29.325539 1547 hybrid_clock.cc:648] HybridClock initialized: now 1786346249323898 us; error 1646 us; skew 500 ppm
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:29.328655 1547 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:29.329664 1547 webserver.cc:460] Webserver started at http://127.12.117.129:41003/ using document root <none> and password file <none>
I20260810 07:17:29.330217 1547 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:29.330281 1547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:29.330500 1547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:29.344192 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "5ea1d7f6a8d0d7d3d3cfcfd05740bb34"
server_key_iv: "a80c0350411b3c8a77401136f101fd09"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.344928 1547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "5ea1d7f6a8d0d7d3d3cfcfd05740bb34"
server_key_iv: "a80c0350411b3c8a77401136f101fd09"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.350349 1547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 07:17:29.368271 1576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.369867 1547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:17:29.369987 1547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "5ea1d7f6a8d0d7d3d3cfcfd05740bb34"
server_key_iv: "a80c0350411b3c8a77401136f101fd09"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.370117 1547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:29.404656 1547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:29.407979 1547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:29.408157 1547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:29.408737 1547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:29.409564 1547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:29.409621 1547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.409672 1547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:29.409695 1547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.419425 1547 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:45021
I20260810 07:17:29.419534 1798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:45021 every 8 connection(s)
I20260810 07:17:29.420336 1547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:29.424778 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 1547
I20260810 07:17:29.424901 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:29.425225 12758 external_mini_cluster.cc:1428] Setting key 748bfddc82fafdf9f9e5e5fa7d6a911e
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:29.438196 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.422799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.129:44169)
0810 07:17:29.423953 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:29.424007 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:29.425008 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:29.425956 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:29.425966 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:29.426292 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:29.426932 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:29.426950 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.428153 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.428156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:29.429510 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:29.429521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.430258 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.430262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:29.430447 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:29.431357 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:29.431376 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:29.433055 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:29.435243 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:29.435249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:29.435291 (+ 42us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:29.435605 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:29.435893 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:29.435897 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:29.435899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:29.436056 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:29.436516 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:29.436522 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:29.436832 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 07:17:29.437004 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 07:17:29.437289 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"thread_start_us":133,"threads_started":1}
I20260810 07:17:29.438944 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.422355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:44169 (local address 127.12.117.190:35159)
0810 07:17:29.425436 (+ 3081us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:29.425444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:29.425461 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:17:29.425505 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:29.425508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:29.425558 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:29.425652 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:29.427094 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.427988 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.429654 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.430005 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.430099 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:29.433249 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:29.433253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:29.433256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:29.433284 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:29.435116 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:29.435711 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:29.435714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:29.435716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:29.435803 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:29.436162 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:29.436166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:29.436167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:29.436381 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:29.436464 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:29.438661 (+ 2197us) server_negotiation.cc:299] Negotiation successful
0810 07:17:29.438781 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2990,"thread_start_us":120,"threads_started":1}
I20260810 07:17:29.440598 1800 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
I20260810 07:17:29.440837 1800 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:29.441395 1800 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:29.443017 1398 ts_manager.cc:194] Registered new tserver with Master: 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129:45021)
I20260810 07:17:29.444591 1398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:44169
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:17:29.472589 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 07:17:29.627895 1808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:29.628223 1808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:29.628290 1808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:29.634020 1808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:29.634100 1808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:29.634182 1808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:29.641592 1808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:29.642881 1808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:29.644239 1808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:29.656814 1819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:29.656939 1821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:29.657305 1808 server_base.cc:1034] running on GCE node
W20260810 07:17:29.657444 1823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:29.657929 1808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:29.658926 1808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:29.660138 1808 hybrid_clock.cc:648] HybridClock initialized: now 1786346249660096 us; error 53 us; skew 500 ppm
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:29.663849 1808 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:29.664949 1808 webserver.cc:460] Webserver started at http://127.12.117.130:42603/ using document root <none> and password file <none>
I20260810 07:17:29.665660 1808 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:29.665761 1808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:29.666100 1808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:29.693004 1808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "85377d94039048348e327069b1fa98bf"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "bb3297832a809c10bf923b1f40b5c751"
server_key_iv: "95125dfa7f7ceca6dfba6d181fd442ac"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.693835 1808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "85377d94039048348e327069b1fa98bf"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "bb3297832a809c10bf923b1f40b5c751"
server_key_iv: "95125dfa7f7ceca6dfba6d181fd442ac"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.699115 1808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 07:17:29.729018 1836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.730784 1808 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260810 07:17:29.730916 1808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "85377d94039048348e327069b1fa98bf"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "bb3297832a809c10bf923b1f40b5c751"
server_key_iv: "95125dfa7f7ceca6dfba6d181fd442ac"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:29.731040 1808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:29.751514 1808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:29.755038 1808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:29.755298 1808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:29.756132 1808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:29.757367 1808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:29.757462 1808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.757539 1808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:29.757581 1808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:29.768496 1808 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:36221
I20260810 07:17:29.768716 2030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:36221 every 8 connection(s)
I20260810 07:17:29.769405 1808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:29.772832 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 1808
I20260810 07:17:29.772954 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:29.773273 12758 external_mini_cluster.cc:1428] Setting key 9118bda900aab63a95b811356a9fed7b
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:29.783247 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.771364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:43733 (local address 127.12.117.190:35159)
0810 07:17:29.771486 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:29.771489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:29.772447 (+ 958us) server_negotiation.cc:407] Connection header received
0810 07:17:29.773668 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:29.773673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:29.773735 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:29.773814 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:29.775130 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.775891 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.777330 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.777568 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.777658 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:29.779934 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:29.779939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:29.779942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:29.779969 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:29.781314 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:29.781784 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:29.781788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:29.781790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:29.781883 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:29.782200 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:29.782203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:29.782205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:29.782386 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:29.782473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:29.782949 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 07:17:29.783080 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:17:29.784088 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:29.771820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.130:43733)
0810 07:17:29.772299 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:29.772340 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:29.773359 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:29.774103 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:29.774112 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:29.774372 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:29.774960 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:29.774980 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.776034 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.776038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:29.777207 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:29.777219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:29.777714 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:29.777718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:29.777893 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:29.778411 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:29.778427 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:29.779776 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:29.781426 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:29.781432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:29.781446 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:29.781689 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:29.781967 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:29.781971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:29.781973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:29.782102 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:29.782489 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:29.782496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:29.782725 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 07:17:29.782888 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 07:17:29.783197 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":148,"threads_started":1}
I20260810 07:17:29.785223 2031 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:17:29.785519 2031 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:29.785991 2031 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:29.786955 1398 ts_manager.cc:194] Registered new tserver with Master: 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:29.787603 1398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:43733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:17:29.814888 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:29 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346249, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
W20260810 07:17:29.973526 2040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:29.973850 2040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:29.973907 2040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:29.979784 2040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:29.979916 2040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:29.980043 2040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:17:29.987542 2040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:29.989050 2040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:29.990327 2040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:30.003579 2047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.003829 2052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.003593 2046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.004534 2040 server_base.cc:1034] running on GCE node
I20260810 07:17:30.005055 2040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:30.005823 2040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:30.007001 2040 hybrid_clock.cc:648] HybridClock initialized: now 1786346250006975 us; error 41 us; skew 500 ppm
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:30.011008 2040 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:17:30.012089 2040 webserver.cc:460] Webserver started at http://127.12.117.131:36415/ using document root <none> and password file <none>
I20260810 07:17:30.012799 2040 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:30.012912 2040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:30.013186 2040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:30.031399 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "6f9232c4c9c54bd38d868f0eda15fb4a"
format_stamp: "Formatted at 2026-08-10 07:17:30 on dist-test-slave-gjrz"
server_key: "7e296a557927e8830e9632e0868b5fa0"
server_key_iv: "6da00c87381d6eb254189af191b475ec"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.032356 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "6f9232c4c9c54bd38d868f0eda15fb4a"
format_stamp: "Formatted at 2026-08-10 07:17:30 on dist-test-slave-gjrz"
server_key: "7e296a557927e8830e9632e0868b5fa0"
server_key_iv: "6da00c87381d6eb254189af191b475ec"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.037384 2040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 07:17:30.055011 2068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.056613 2040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:30.056738 2040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "6f9232c4c9c54bd38d868f0eda15fb4a"
format_stamp: "Formatted at 2026-08-10 07:17:30 on dist-test-slave-gjrz"
server_key: "7e296a557927e8830e9632e0868b5fa0"
server_key_iv: "6da00c87381d6eb254189af191b475ec"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.056876 2040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:30.076344 2040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:30.079926 2040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:30.080127 2040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:30.080773 2040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:30.081801 2040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:30.081861 2040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.082041 2040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:30.082089 2040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.096041 2040 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:38583
I20260810 07:17:30.096328 2260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:38583 every 8 connection(s)
I20260810 07:17:30.097182 2040 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:30.106434 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 2040
I20260810 07:17:30.106544 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:17:30.106846 12758 external_mini_cluster.cc:1428] Setting key 5403407f530dc2a924bc18caaca1758a
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:30.116521 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.100973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:55589 (local address 127.12.117.190:35159)
0810 07:17:30.101115 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:30.101120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:30.102274 (+ 1154us) server_negotiation.cc:407] Connection header received
0810 07:17:30.103783 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:30.103787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:30.103848 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:30.103943 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:30.105658 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.106487 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.108332 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.108676 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.108761 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.112377 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:30.112382 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:30.112384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:30.112413 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:30.114139 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.114751 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.114754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.114756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.114855 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.115225 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.115229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.115232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.115507 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:30.115608 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:30.116211 (+ 603us) server_negotiation.cc:299] Negotiation successful
0810 07:17:30.116348 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:30.117398 2264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.101471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.131:55589)
0810 07:17:30.102111 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.102168 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:30.103517 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:30.104165 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:30.104175 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:30.104697 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:30.105476 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.105497 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.106661 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.106665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.108179 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.108192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.108962 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.108966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.109211 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.109955 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:30.109976 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:30.112183 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:30.114264 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.114271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.114285 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.114617 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.114942 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.114945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.114947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.115118 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.115606 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:30.115612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:30.115917 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 07:17:30.116105 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 07:17:30.116407 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":215,"threads_started":1}
I20260810 07:17:30.118568 2261 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
I20260810 07:17:30.118821 2261 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.119436 2261 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:30.120733 1398 ts_manager.cc:194] Registered new tserver with Master: 6f9232c4c9c54bd38d868f0eda15fb4a (127.12.117.131:38583)
I20260810 07:17:30.121512 1398 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:55589
I20260810 07:17:30.132377 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:30.133128 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 1547
I20260810 07:17:30.152006 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 1808
I20260810 07:17:30.161734 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 2040
I20260810 07:17:30.175397 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 1341
I20260810 07:17:30.185855 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:35159
--webserver_interface=127.12.117.190
--webserver_port=41751
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:35159
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:30.295463 2265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:30.295696 2265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:30.295750 2265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:30.299706 2265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:30.299758 2265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:30.299783 2265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:30.299805 2265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:30.299834 2265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:30.305047 2265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:35159
--ranger_config_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=41751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:30.305938 2265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:30.306838 2265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:30.314762 2281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.314775 2275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.314865 2265 server_base.cc:1034] running on GCE node
W20260810 07:17:30.314775 2276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.315668 2265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:30.316648 2265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:30.317803 2265 hybrid_clock.cc:648] HybridClock initialized: now 1786346250317781 us; error 38 us; skew 500 ppm
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:30.320256 2265 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:30.321075 2265 webserver.cc:460] Webserver started at http://127.12.117.190:41751/ using document root <none> and password file <none>
I20260810 07:17:30.321574 2265 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:30.321645 2265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:30.392861 2265 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:30.394891 2304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.396152 2265 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:30.396265 2265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "a79767011e7146d19ecd4c011df6983f"
format_stamp: "Formatted at 2026-08-10 07:17:28 on dist-test-slave-gjrz"
server_key: "aeac024ecdf09e242f8e0ed9e799ceba"
server_key_iv: "8d3350f2851751edbfc00c9912dfa696"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.396783 2265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:30.412668 2265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:30.414891 2265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:30.415057 2265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:30.421602 2265 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:35159
I20260810 07:17:30.421634 2403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:35159 every 8 connection(s)
I20260810 07:17:30.422566 2265 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:30.426584 2404 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:30.428260 2404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Bootstrap starting.
I20260810 07:17:30.432507 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 2265
I20260810 07:17:30.433007 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:45021
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:35159
--webserver_port=41003
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:30.444785 2404 log.cc:824] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:30.451145 2404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:30.451503 2404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Bootstrap complete.
I20260810 07:17:30.455296 2404 raft_consensus.cc:348] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:30.455701 2404 raft_consensus.cc:729] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79767011e7146d19ecd4c011df6983f, State: Initialized, Role: FOLLOWER
I20260810 07:17:30.456204 2404 consensus_queue.cc:260] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:30.456323 2404 raft_consensus.cc:388] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:30.456386 2404 raft_consensus.cc:482] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:30.456480 2404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:17:30.458287 2404 raft_consensus.cc:504] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:30.458583 2404 leader_election.cc:302] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a79767011e7146d19ecd4c011df6983f; no voters:
I20260810 07:17:30.458848 2404 leader_election.cc:288] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:17:30.458966 2409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:17:30.459192 2409 raft_consensus.cc:686] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [term 2 LEADER]: Becoming Leader. State: Replica: a79767011e7146d19ecd4c011df6983f, State: Running, Role: LEADER
I20260810 07:17:30.459646 2409 consensus_queue.cc:237] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } }
I20260810 07:17:30.459970 2404 sys_catalog.cc:564] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:30.460580 2409 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a79767011e7146d19ecd4c011df6983f. Latest consensus state: current_term: 2 leader_uuid: "a79767011e7146d19ecd4c011df6983f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } } }
I20260810 07:17:30.460693 2409 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:30.461026 2428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:30.461117 2411 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a79767011e7146d19ecd4c011df6983f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a79767011e7146d19ecd4c011df6983f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 35159 } } }
I20260810 07:17:30.461282 2411 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:30.464133 2428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:30.464622 2404 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 07:17:30.464761 2428 catalog_manager.cc:1259] Loaded cluster ID: a4d3e028dc9b4349be68e1dc343db2f0
I20260810 07:17:30.464808 2428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:30.467092 2428 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:30.467767 2428 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Loaded TSK: 0
I20260810 07:17:30.468928 2428 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:17:30.574031 2406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:30.574323 2406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:30.574381 2406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:30.579958 2406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:30.580029 2406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:30.580116 2406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:30.587591 2406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:45021
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=41003
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:30.588742 2406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:30.590040 2406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:30.602187 2479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.602313 2406 server_base.cc:1034] running on GCE node
W20260810 07:17:30.602205 2477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.602648 2476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.603233 2406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:30.604024 2406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:30.605214 2406 hybrid_clock.cc:648] HybridClock initialized: now 1786346250605184 us; error 45 us; skew 500 ppm
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:30.608688 2406 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:30.609762 2406 webserver.cc:460] Webserver started at http://127.12.117.129:41003/ using document root <none> and password file <none>
I20260810 07:17:30.610455 2406 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:30.610561 2406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:30.628594 2406 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:17:30.631904 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.633285 2406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:30.633400 2406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "5ea1d7f6a8d0d7d3d3cfcfd05740bb34"
server_key_iv: "a80c0350411b3c8a77401136f101fd09"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.634094 2406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:30.649118 2406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:30.652833 2406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:30.653064 2406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:30.653738 2406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:30.654772 2406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:30.654825 2406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.654874 2406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:30.654896 2406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.666006 2406 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:45021
I20260810 07:17:30.666213 2692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:45021 every 8 connection(s)
I20260810 07:17:30.667222 2406 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:30.670956 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 2406
I20260810 07:17:30.671418 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:36221
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:35159
--webserver_port=42603
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:30.688316 2698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.669761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.129:52045)
0810 07:17:30.671939 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.671998 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:30.672943 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:30.674278 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:30.674286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:30.674591 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:30.675176 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.675198 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.677180 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.677183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.678564 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.678578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.679888 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.679893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.680083 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.680715 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:30.680737 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:30.682236 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:30.684731 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.684737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.684751 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.685030 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.685438 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.685441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.685443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.685624 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.686443 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:30.686450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:30.686754 (+ 304us) client_negotiation.cc:769] Sending connection context
0810 07:17:30.686963 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 07:17:30.687309 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1977,"thread_start_us":124,"threads_started":1}
I20260810 07:17:30.690053 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.669595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:52045 (local address 127.12.117.190:35159)
0810 07:17:30.673317 (+ 3722us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:30.673341 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:30.673376 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:17:30.673599 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:30.673609 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:30.673897 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:30.674190 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:30.675441 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.677011 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.678727 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.679479 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.679665 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.682455 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:30.682462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:30.682479 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:30.682511 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:30.684579 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.685170 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.685176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.685182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.685326 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.685742 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.685746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.685748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.686226 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:30.686379 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:30.688856 (+ 2477us) server_negotiation.cc:299] Negotiation successful
0810 07:17:30.689147 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1771,"thread_start_us":125,"threads_started":1}
I20260810 07:17:30.693972 2693 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
I20260810 07:17:30.694203 2693 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.694775 2693 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:30.697532 2693 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:35159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:30.697669 2693 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.697835 2693 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:30.698849 2693 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.699024 2693 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:30.699920 2693 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:30.821400 2699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:30.821620 2699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:30.821660 2699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:30.825745 2699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:30.825798 2699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:30.825871 2699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:30.831353 2699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:36221
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=42603
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:30.832334 2699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:30.833277 2699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:30.842666 2708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.842669 2712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:30.842669 2706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:30.843122 2699 server_base.cc:1034] running on GCE node
I20260810 07:17:30.843514 2699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:30.844092 2699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:30.845252 2699 hybrid_clock.cc:648] HybridClock initialized: now 1786346250845219 us; error 49 us; skew 500 ppm
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:30.848733 2699 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:30.849742 2699 webserver.cc:460] Webserver started at http://127.12.117.130:42603/ using document root <none> and password file <none>
I20260810 07:17:30.850394 2699 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:30.850504 2699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:30.870572 2699 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:30.873633 2729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.875200 2699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:17:30.875355 2699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "85377d94039048348e327069b1fa98bf"
format_stamp: "Formatted at 2026-08-10 07:17:29 on dist-test-slave-gjrz"
server_key: "bb3297832a809c10bf923b1f40b5c751"
server_key_iv: "95125dfa7f7ceca6dfba6d181fd442ac"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:30.876041 2699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:30.895696 2699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:30.899050 2699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:30.899309 2699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:30.900003 2699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:30.901111 2699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:30.901183 2699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.901310 2699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:30.901348 2699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:30.911906 2699 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:36221
I20260810 07:17:30.912873 2699 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:30.913734 2918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:36221 every 8 connection(s)
I20260810 07:17:30.922292 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 2699
I20260810 07:17:30.922801 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.117.148:44797/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:38583
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:35159
--webserver_port=36415
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:40535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:30 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346250, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:30.934365 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.914911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:60443 (local address 127.12.117.190:35159)
0810 07:17:30.915060 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:30.915065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:30.915938 (+ 873us) server_negotiation.cc:407] Connection header received
0810 07:17:30.916842 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:30.916846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:30.916906 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:30.917006 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:30.924234 (+ 7228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.925077 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.927090 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.927490 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.927577 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.930276 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:30.930280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:30.930283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:30.930311 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:30.931967 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.932586 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.932590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.932591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.932680 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:30.933081 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:30.933084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:30.933086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:30.933310 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:30.933398 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:30.934062 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 07:17:30.934197 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:17:30.936317 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:30.915314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.130:60443)
0810 07:17:30.915775 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:30.915826 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:30.916632 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:30.922708 (+ 6076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:30.922718 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:30.923330 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:30.924020 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.924043 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.925249 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.925252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.926942 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:30.926955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:30.927791 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:30.927795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:30.927991 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:30.928575 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:30.928596 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:30.930080 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:30.932100 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.932106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.932121 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.932463 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.932793 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:30.932795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:30.932797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:30.932968 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:30.933486 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:30.933492 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:30.933784 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 07:17:30.934979 (+ 1195us) client_negotiation.cc:240] Negotiation successful
0810 07:17:30.935327 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260810 07:17:30.937839 2919 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
I20260810 07:17:30.938098 2919 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.938683 2919 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:30.940461 2919 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:35159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:30.940574 2919 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.940718 2919 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:30.941586 2919 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:30.941737 2919 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:30.942518 2919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:31.086449 2925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:31.086779 2925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:31.086846 2925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:31.092608 2925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:31.092680 2925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:31.092763 2925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:17:31.100229 2925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:40535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.117.148:44797/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:38583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=36415
--tserver_master_addrs=127.12.117.190:35159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:31.101511 2925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:31.102835 2925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:31.115834 2941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:31.115911 2938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:31.116155 2939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:31.116847 2925 server_base.cc:1034] running on GCE node
I20260810 07:17:31.117321 2925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:31.118112 2925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:31.119326 2925 hybrid_clock.cc:648] HybridClock initialized: now 1786346251119302 us; error 46 us; skew 500 ppm
Aug 10 07:17:31 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:31.123027 2925 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:17:31.124182 2925 webserver.cc:460] Webserver started at http://127.12.117.131:36415/ using document root <none> and password file <none>
I20260810 07:17:31.124938 2925 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:31.125115 2925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 07:17:31 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:31.142822 2925 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 07:17:31.146042 2955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:31.147732 2925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 07:17:31.147857 2925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "6f9232c4c9c54bd38d868f0eda15fb4a"
format_stamp: "Formatted at 2026-08-10 07:17:30 on dist-test-slave-gjrz"
server_key: "7e296a557927e8830e9632e0868b5fa0"
server_key_iv: "6da00c87381d6eb254189af191b475ec"
server_key_version: "kuduclusterkey@0"
I20260810 07:17:31.148563 2925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:31.175415 2925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:31.183753 2925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:31.183976 2925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:31.184666 2925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:31.185691 2925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:31.185779 2925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:31.185851 2925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:31.185890 2925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:31.198889 2925 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:38583
I20260810 07:17:31.200284 2925 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:31.200985 3153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:38583 every 8 connection(s)
I20260810 07:17:31.201910 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 2925
I20260810 07:17:31.223228 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.202600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.12.117.190:35159)
0810 07:17:31.202948 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:31.202953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:31.206924 (+ 3971us) server_negotiation.cc:407] Connection header received
0810 07:17:31.208749 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:31.208752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:31.208803 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:31.208850 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:31.209844 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.210660 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.217407 (+ 6747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.217759 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.217900 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:31.219583 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:31.219587 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:31.219591 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:31.219623 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:31.221351 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:31.221825 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:31.221829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:31.221831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:31.221954 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:31.222252 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:31.222256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:31.222258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:31.222509 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:31.222722 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:31.222905 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 07:17:31.223042 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":245,"spinlock_wait_cycles":896}
Aug 10 07:17:31 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:31.233723 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.206197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:36463 (local address 127.12.117.190:35159)
0810 07:17:31.207463 (+ 1266us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:31.207477 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:31.207507 (+ 30us) server_negotiation.cc:407] Connection header received
0810 07:17:31.208622 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:31.208626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:31.208690 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:31.208805 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:31.218803 (+ 9998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.220090 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.226328 (+ 6238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.226699 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.226771 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:31.229629 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:31.229633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:31.229636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:31.229663 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:31.231327 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:31.231936 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:31.231941 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:31.231943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:31.232051 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:31.232444 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:31.232448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:31.232450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:31.232697 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:31.232798 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:31.233400 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 07:17:31.233538 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":1179,"spinlock_wait_cycles":41600,"thread_start_us":67,"threads_started":1}
I20260810 07:17:31.234599 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:31.205175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:35159 (local address 127.12.117.131:36463)
0810 07:17:31.207353 (+ 2178us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:31.207404 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:31.208395 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:31.209248 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:31.209258 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:31.209623 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:31.218614 (+ 8991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:31.218641 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.224209 (+ 5568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.224212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:31.226181 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:31.226195 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:31.226967 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:31.226971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:31.227179 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:31.227793 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:31.227816 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:31.229417 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:31.231463 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:31.231469 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:31.231486 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:31.231807 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:31.232145 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:31.232148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:31.232149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:31.232319 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:31.232801 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:31.232808 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:31.233121 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 07:17:31.233314 (+ 193us) client_negotiation.cc:240] Negotiation successful
0810 07:17:31.233627 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1934,"thread_start_us":128,"threads_started":1}
I20260810 07:17:31.236079 3154 heartbeater.cc:347] Connected to a master server at 127.12.117.190:35159
I20260810 07:17:31.236322 3154 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:31.236878 3154 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:31.238621 3154 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:35159 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 07:17:31.238737 3154 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:31.238893 3154 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:31.239838 3154 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:31.240022 3154 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
W20260810 07:17:31.240938 3154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 07:17:31 dist-test-slave-gjrz krb5kdc[567](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:31.700135 2693 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:31.700400 2693 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:31.701509 2339 ts_manager.cc:194] Registered new tserver with Master: 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129:45021)
I20260810 07:17:31.703286 2339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:52045
Aug 10 07:17:31 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:31.942723 2919 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:31.942989 2919 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:31.943737 2339 ts_manager.cc:194] Registered new tserver with Master: 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:31.944535 2339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:60443
Aug 10 07:17:32 dist-test-slave-gjrz krb5kdc[567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346251, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for HTTP/127.12.117.148@KRBTEST.COM
I20260810 07:17:32.241151 3154 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:32.241441 3154 heartbeater.cc:511] Master 127.12.117.190:35159 requested a full tablet report, sending...
I20260810 07:17:32.242138 2339 ts_manager.cc:194] Registered new tserver with Master: 6f9232c4c9c54bd38d868f0eda15fb4a (127.12.117.131:38583)
I20260810 07:17:32.242924 2339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:36463
I20260810 07:17:32.251021 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:32.271413 3186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.254063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.12.117.190:35159)
0810 07:17:32.255249 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.255254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.257773 (+ 2519us) server_negotiation.cc:407] Connection header received
0810 07:17:32.257818 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.257821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.257870 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.257970 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:32.258893 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.259487 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.261635 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.262346 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.262457 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.264817 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:32.264821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:32.264824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:32.264851 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:32.266726 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:32.267213 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:32.267217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:32.267218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:32.267340 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:32.267668 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:32.267672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:32.267673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:32.267899 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:32.271012 (+ 3113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.271094 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.271215 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":69,"threads_started":1}
I20260810 07:17:32.275283 2339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:32.278605 2339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:32.321530 3199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.302888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38583 (local address 127.0.0.1:51880)
0810 07:17:32.315589 (+ 12701us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.315656 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.315811 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.316621 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.316628 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.316652 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.316794 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.316800 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.318266 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.318269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.319680 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.319692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.320980 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.320984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.321098 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.321137 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.321242 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.321358 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12476,"thread_start_us":123,"threads_started":1}
I20260810 07:17:32.322145 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.303108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.12.117.131:38583)
0810 07:17:32.303676 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.303683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.315964 (+ 12281us) server_negotiation.cc:407] Connection header received
0810 07:17:32.316052 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.316058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.316284 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.316440 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.316924 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.318139 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.319820 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.320662 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.320775 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.320913 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.321895 (+ 982us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.322003 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":63,"threads_started":1}
I20260810 07:17:32.325762 3047 tablet_service.cc:1467] Processing CreateTablet for tablet caf7ac40d1f247e6805c7032b7a036ab (DEFAULT_TABLE table=test-table [id=5a47e6a956024453b905995994c66117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:32.326948 3047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf7ac40d1f247e6805c7032b7a036ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:32.327394 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.304911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36221 (local address 127.0.0.1:41218)
0810 07:17:32.320837 (+ 15926us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.320851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.321084 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.322483 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.322486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.322505 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.322594 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.322600 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.324499 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.324503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.325685 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.325698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.326980 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.326984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.327071 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.327097 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.327168 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.327235 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15860,"thread_start_us":137,"threads_started":1}
I20260810 07:17:32.328722 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.305390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.12.117.130:36221)
0810 07:17:32.305690 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.305696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.320968 (+ 15272us) server_negotiation.cc:407] Connection header received
0810 07:17:32.321973 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.321980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.322179 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.322323 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.322728 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.324368 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.325830 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.326657 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.326775 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.326900 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.327628 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.327742 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":133,"threads_started":1}
I20260810 07:17:32.331667 2815 tablet_service.cc:1467] Processing CreateTablet for tablet caf7ac40d1f247e6805c7032b7a036ab (DEFAULT_TABLE table=test-table [id=5a47e6a956024453b905995994c66117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:32.332787 2815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf7ac40d1f247e6805c7032b7a036ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:32.332895 3195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.304671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45021 (local address 127.0.0.1:39886)
0810 07:17:32.319733 (+ 15062us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.319748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.319978 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.323008 (+ 3030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.323012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.323032 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.323113 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.323119 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.330327 (+ 7208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.330331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.331469 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.331481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.332542 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.332546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.332633 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.332659 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.332707 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.332771 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14989,"thread_start_us":92,"threads_started":1}
I20260810 07:17:32.333490 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.305265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.12.117.129:45021)
0810 07:17:32.321358 (+ 16093us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.321366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.321389 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:17:32.321479 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.321485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.321716 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.323295 (+ 1579us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.329080 (+ 5785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.330184 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.331633 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.332387 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.333082 (+ 695us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.333210 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.333260 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.333345 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15972,"thread_start_us":115,"threads_started":1}
I20260810 07:17:32.336306 2548 tablet_service.cc:1467] Processing CreateTablet for tablet caf7ac40d1f247e6805c7032b7a036ab (DEFAULT_TABLE table=test-table [id=5a47e6a956024453b905995994c66117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:32.337375 2548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caf7ac40d1f247e6805c7032b7a036ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:32.345548 3207 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Bootstrap starting.
I20260810 07:17:32.349115 3207 tablet_bootstrap.cc:654] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:32.350023 3207 log.cc:824] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:32.351184 3208 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Bootstrap starting.
I20260810 07:17:32.352926 3207 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: No bootstrap required, opened a new log
I20260810 07:17:32.353139 3207 ts_tablet_manager.cc:1397] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260810 07:17:32.353881 3208 tablet_bootstrap.cc:654] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:32.355420 3206 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Bootstrap starting.
I20260810 07:17:32.357484 3207 raft_consensus.cc:348] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.357842 3207 raft_consensus.cc:374] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:32.357937 3207 raft_consensus.cc:729] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ec3b54fb0b34873bd92d1aa62d5d7d4, State: Initialized, Role: FOLLOWER
I20260810 07:17:32.358490 3208 log.cc:824] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:32.358476 3207 consensus_queue.cc:260] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.359102 3206 tablet_bootstrap.cc:654] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:32.360633 3206 log.cc:824] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:32.360901 3207 ts_tablet_manager.cc:1428] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260810 07:17:32.361325 2693 heartbeater.cc:503] Master 127.12.117.190:35159 was elected leader, sending a full tablet report...
I20260810 07:17:32.365703 3208 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: No bootstrap required, opened a new log
I20260810 07:17:32.365906 3208 ts_tablet_manager.cc:1397] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260810 07:17:32.373867 3208 raft_consensus.cc:348] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.374146 3208 raft_consensus.cc:374] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:32.374192 3208 raft_consensus.cc:729] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f9232c4c9c54bd38d868f0eda15fb4a, State: Initialized, Role: FOLLOWER
I20260810 07:17:32.374648 3208 consensus_queue.cc:260] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.375696 3208 ts_tablet_manager.cc:1428] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20260810 07:17:32.376032 3154 heartbeater.cc:503] Master 127.12.117.190:35159 was elected leader, sending a full tablet report...
I20260810 07:17:32.380089 3206 tablet_bootstrap.cc:492] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: No bootstrap required, opened a new log
I20260810 07:17:32.380306 3206 ts_tablet_manager.cc:1397] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260810 07:17:32.386119 3206 raft_consensus.cc:348] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.386386 3206 raft_consensus.cc:374] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:32.386451 3206 raft_consensus.cc:729] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85377d94039048348e327069b1fa98bf, State: Initialized, Role: FOLLOWER
I20260810 07:17:32.386919 3206 consensus_queue.cc:260] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.387919 3206 ts_tablet_manager.cc:1428] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260810 07:17:32.388276 2919 heartbeater.cc:503] Master 127.12.117.190:35159 was elected leader, sending a full tablet report...
I20260810 07:17:32.389763 3221 raft_consensus.cc:482] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:32.389938 3221 raft_consensus.cc:504] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.390447 3215 raft_consensus.cc:482] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:32.390614 3215 raft_consensus.cc:504] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.391106 3221 leader_election.cc:288] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129:45021), 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:32.392828 3215 leader_election.cc:288] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f9232c4c9c54bd38d868f0eda15fb4a (127.12.117.131:38583), 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:32.399983 3196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.392157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:53109 (local address 127.12.117.131:38583)
0810 07:17:32.392257 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.392262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.393246 (+ 984us) server_negotiation.cc:407] Connection header received
0810 07:17:32.393334 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.393337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.393378 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.393490 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.396354 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.397064 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.398577 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.399156 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.399253 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.399327 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.399744 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.399844 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:17:32.400518 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.391783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:53719 (local address 127.12.117.129:45021)
0810 07:17:32.393556 (+ 1773us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.393561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.393576 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:17:32.393621 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.393623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.393664 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.393734 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.395595 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.396305 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.398571 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.399167 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.399253 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.399324 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.400324 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.400397 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1690}
I20260810 07:17:32.400414 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.391781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:38583 (local address 127.12.117.129:53109)
0810 07:17:32.393158 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.393168 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.393259 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.393607 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.393610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.393625 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.396216 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.396225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.397207 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.397210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.398423 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.398437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.399429 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.399433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.399545 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.399569 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.400188 (+ 619us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.400267 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"thread_start_us":160,"threads_started":1}
I20260810 07:17:32.400676 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.391465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45021 (local address 127.12.117.131:53719)
0810 07:17:32.392569 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.392585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.392678 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.395317 (+ 2639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.395320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.395339 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.395467 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.395473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.397142 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.397145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.398422 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.398433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.400095 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.400099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.400196 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.400218 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.400443 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.400520 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1038,"thread_start_us":107,"threads_started":1}
I20260810 07:17:32.401208 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" is_pre_election: true
I20260810 07:17:32.401412 2599 raft_consensus.cc:2454] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f9232c4c9c54bd38d868f0eda15fb4a in term 0.
I20260810 07:17:32.401815 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" is_pre_election: true
I20260810 07:17:32.402019 3077 raft_consensus.cc:2454] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ec3b54fb0b34873bd92d1aa62d5d7d4 in term 0.
I20260810 07:17:32.402441 2494 leader_election.cc:302] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec3b54fb0b34873bd92d1aa62d5d7d4, 6f9232c4c9c54bd38d868f0eda15fb4a; no voters:
I20260810 07:17:32.402745 3215 raft_consensus.cc:2781] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:32.402827 3215 raft_consensus.cc:482] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:32.402871 3215 raft_consensus.cc:3037] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:32.404261 3215 raft_consensus.cc:504] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.404611 3215 leader_election.cc:288] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [CANDIDATE]: Term 1 election: Requested vote from peers 6f9232c4c9c54bd38d868f0eda15fb4a (127.12.117.131:38583), 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:32.405042 2958 leader_election.cc:302] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ec3b54fb0b34873bd92d1aa62d5d7d4, 6f9232c4c9c54bd38d868f0eda15fb4a; no voters:
I20260810 07:17:32.405313 3221 raft_consensus.cc:2781] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:32.405387 3221 raft_consensus.cc:482] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:32.405437 3221 raft_consensus.cc:3037] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:32.405833 3225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.392567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36221 (local address 127.12.117.129:58821)
0810 07:17:32.393430 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.393442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.393510 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.401719 (+ 8209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.401723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.401753 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.401995 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.402005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.403157 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.403161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.404461 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.404473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.405452 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.405455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.405551 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.405572 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.405631 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.405700 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"thread_start_us":133,"threads_started":1}
I20260810 07:17:32.405861 3203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.393969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35605 (local address 127.12.117.130:36221)
0810 07:17:32.394208 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.394212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.394226 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:17:32.394268 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.394270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.394311 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.394394 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.394816 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.396229 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.404671 (+ 8442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.405229 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.405319 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.405395 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.405685 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.405758 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":164}
I20260810 07:17:32.406189 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.392053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36221 (local address 127.12.117.131:35605)
0810 07:17:32.392856 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.392868 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:32.392962 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:32.394506 (+ 1544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:32.394509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:32.394527 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:32.394675 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.394684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.396830 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.396833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.404549 (+ 7716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:32.404560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.405447 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.405450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:32.405551 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.405572 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:17:32.405979 (+ 407us) client_negotiation.cc:240] Negotiation successful
0810 07:17:32.406058 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":93,"threads_started":1}
I20260810 07:17:32.406381 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.394059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:58821 (local address 127.12.117.130:36221)
0810 07:17:32.401298 (+ 7239us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.401305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.401319 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:17:32.401359 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.401362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.401404 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.401482 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:32.402148 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.403025 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.404612 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.405271 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.406104 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.406150 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.406186 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.406248 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":7171,"thread_start_us":116,"threads_started":1}
I20260810 07:17:32.406591 3221 raft_consensus.cc:504] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.406944 3221 leader_election.cc:288] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [CANDIDATE]: Term 1 election: Requested vote from peers 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129:45021), 85377d94039048348e327069b1fa98bf (127.12.117.130:36221)
I20260810 07:17:32.407045 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377d94039048348e327069b1fa98bf"
I20260810 07:17:32.407084 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377d94039048348e327069b1fa98bf" is_pre_election: true
I20260810 07:17:32.407234 2824 raft_consensus.cc:3037] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:32.407512 2844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377d94039048348e327069b1fa98bf" is_pre_election: true
I20260810 07:17:32.407521 2843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85377d94039048348e327069b1fa98bf"
I20260810 07:17:32.408115 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a"
I20260810 07:17:32.408145 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caf7ac40d1f247e6805c7032b7a036ab" candidate_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4"
I20260810 07:17:32.408234 3077 raft_consensus.cc:2379] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4ec3b54fb0b34873bd92d1aa62d5d7d4 in current term 1: Already voted for candidate 6f9232c4c9c54bd38d868f0eda15fb4a in this term.
I20260810 07:17:32.408296 2599 raft_consensus.cc:2379] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f9232c4c9c54bd38d868f0eda15fb4a in current term 1: Already voted for candidate 4ec3b54fb0b34873bd92d1aa62d5d7d4 in this term.
I20260810 07:17:32.408520 2824 raft_consensus.cc:2454] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ec3b54fb0b34873bd92d1aa62d5d7d4 in term 1.
I20260810 07:17:32.408632 2958 leader_election.cc:302] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f9232c4c9c54bd38d868f0eda15fb4a; no voters: 4ec3b54fb0b34873bd92d1aa62d5d7d4, 85377d94039048348e327069b1fa98bf
I20260810 07:17:32.408800 3221 raft_consensus.cc:2726] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 07:17:32.408833 2496 leader_election.cc:302] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4ec3b54fb0b34873bd92d1aa62d5d7d4, 85377d94039048348e327069b1fa98bf; no voters: 6f9232c4c9c54bd38d868f0eda15fb4a
I20260810 07:17:32.408985 3215 raft_consensus.cc:2781] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:32.409178 3215 raft_consensus.cc:686] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [term 1 LEADER]: Becoming Leader. State: Replica: 4ec3b54fb0b34873bd92d1aa62d5d7d4, State: Running, Role: LEADER
I20260810 07:17:32.409488 3215 consensus_queue.cc:237] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } }
I20260810 07:17:32.413030 2339 catalog_manager.cc:5495] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ec3b54fb0b34873bd92d1aa62d5d7d4" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 45021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 } health_report { overall_health: UNKNOWN } } }
W20260810 07:17:32.415424 2923 tablet.cc:2369] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:17:32.419821 2694 tablet.cc:2369] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:32.439381 3201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.434342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.12.117.129:45021)
0810 07:17:32.434460 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:32.434465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:32.434479 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:17:32.434555 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:32.434558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:32.434625 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:32.434729 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:32.435128 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.435922 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.437139 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:32.437469 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:32.437568 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:32.437995 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:32.438573 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:32.438690 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:32.439145 (+ 455us) server_negotiation.cc:299] Negotiation successful
0810 07:17:32.439201 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:17:32.446439 2824 raft_consensus.cc:1261] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 1 FOLLOWER]: Refusing update from remote peer 4ec3b54fb0b34873bd92d1aa62d5d7d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:32.447096 3215 consensus_queue.cc:1035] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85377d94039048348e327069b1fa98bf" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 36221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:32.449756 3077 raft_consensus.cc:1261] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Refusing update from remote peer 4ec3b54fb0b34873bd92d1aa62d5d7d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:32.450459 3215 consensus_queue.cc:1035] T caf7ac40d1f247e6805c7032b7a036ab P 4ec3b54fb0b34873bd92d1aa62d5d7d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f9232c4c9c54bd38d868f0eda15fb4a" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 38583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 07:17:32.458180 3159 tablet.cc:2369] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:32.467412 3240 mvcc.cc:205] Tried to move back new op lower bound from 7316874250012143616 to 7316874249871831040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874250012143616}]
I20260810 07:17:32.469239 3245 mvcc.cc:205] Tried to move back new op lower bound from 7316874250012143616 to 7316874249871831040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874250012143616}]
I20260810 07:17:32.477386 3241 mvcc.cc:205] Tried to move back new op lower bound from 7316874250012143616 to 7316874249871831040. Current Snapshot: MvccSnapshot[applied={T|T < 7316874250012143616}]
I20260810 07:17:32.480012 2339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:32.480178 2339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:32.482738 2339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a79767011e7146d19ecd4c011df6983f: Sending DeleteTablet for 3 replicas of tablet caf7ac40d1f247e6805c7032b7a036ab
I20260810 07:17:32.484185 3047 tablet_service.cc:1514] Processing DeleteTablet for tablet caf7ac40d1f247e6805c7032b7a036ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:32 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:51880
I20260810 07:17:32.484795 2548 tablet_service.cc:1514] Processing DeleteTablet for tablet caf7ac40d1f247e6805c7032b7a036ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:32 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:39886
I20260810 07:17:32.485316 2815 tablet_service.cc:1514] Processing DeleteTablet for tablet caf7ac40d1f247e6805c7032b7a036ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:32 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:41218
I20260810 07:17:32.486541 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 2406
I20260810 07:17:32.494923 3260 tablet_replica.cc:330] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: stopping tablet replica
I20260810 07:17:32.495172 3263 tablet_replica.cc:330] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: stopping tablet replica
I20260810 07:17:32.495618 3263 raft_consensus.cc:2229] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:32.495976 3260 raft_consensus.cc:2229] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:32.496415 3260 raft_consensus.cc:2258] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:32.496843 3263 raft_consensus.cc:2258] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:32.498795 3260 ts_tablet_manager.cc:1905] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:32.500396 3263 ts_tablet_manager.cc:1905] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:32.503779 3260 ts_tablet_manager.cc:1918] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 07:17:32.505455 2308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:45021 (error 108)
I20260810 07:17:32.506641 3263 ts_tablet_manager.cc:1918] T caf7ac40d1f247e6805c7032b7a036ab P 85377d94039048348e327069b1fa98bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:32.506829 3260 log.cc:1196] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/caf7ac40d1f247e6805c7032b7a036ab
I20260810 07:17:32.506933 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.506590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:45021 (local address 127.0.0.1:39904)
0810 07:17:32.506772 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.506846 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:45021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260810 07:17:32.507110 2308 catalog_manager.cc:4584] TS 4ec3b54fb0b34873bd92d1aa62d5d7d4 (127.12.117.129:45021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet caf7ac40d1f247e6805c7032b7a036ab: Network error: Client connection negotiation failed: client connection to 127.12.117.129:45021: connect: Connection refused (error 111)
I20260810 07:17:32.507450 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 2699
I20260810 07:17:32.510682 3260 ts_tablet_manager.cc:1939] T caf7ac40d1f247e6805c7032b7a036ab P 6f9232c4c9c54bd38d868f0eda15fb4a: Deleting consensus metadata
I20260810 07:17:32.540230 2308 catalog_manager.cc:4841] TS 6f9232c4c9c54bd38d868f0eda15fb4a (127.12.117.131:38583): tablet caf7ac40d1f247e6805c7032b7a036ab (table test-table [id=5a47e6a956024453b905995994c66117]) successfully deleted
I20260810 07:17:32.546753 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 2925
I20260810 07:17:32.553980 3198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:32.553628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:36221 (local address 127.0.0.1:41222)
0810 07:17:32.553827 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:32.553882 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.130:36221: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 07:17:32.559602 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 2265
I20260810 07:17:32.573835 12758 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 07:17:32.918045 12758 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 07:17:32.918160 12758 mini_ranger.cc:66] Stopping Ranger...
I20260810 07:17:33.261667 12758 mini_ranger.cc:68] Stopped Ranger
2026-08-10 07:17:33.262 UTC [615] LOG: received smart shutdown request
2026-08-10 07:17:33.265 UTC [615] LOG: background worker "logical replication launcher" (PID 627) exited with exit code 1
2026-08-10 07:17:33.265 UTC [620] LOG: shutting down
2026-08-10 07:17:33.400 UTC [615] LOG: database system is shut down
2026-08-10T07:17:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45776 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:33 dist-test-slave-gjrz krb5kdc[3276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:33 dist-test-slave-gjrz krb5kdc[3276](info): set up 2 sockets
Aug 10 07:17:33 dist-test-slave-gjrz krb5kdc[3276](info): commencing operation
krb5kdc: starting...
W20260810 07:17:34.664618 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:17:34 dist-test-slave-gjrz krb5kdc[3276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:17:34.777248 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44403
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:34.883626 3296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:34.883857 3296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:34.883913 3296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:34.887776 3296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:34.887825 3296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:34.887851 3296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:34.887872 3296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:34.887902 3296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:34.892848 3296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:34.893738 3296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:34.894665 3296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:34.902631 3305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:34.902686 3307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:34.902665 3304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:34.902781 3296 server_base.cc:1034] running on GCE node
I20260810 07:17:34.903512 3296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:34.904557 3296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:34.905705 3296 hybrid_clock.cc:648] HybridClock initialized: now 1786346254905684 us; error 33 us; skew 500 ppm
Aug 10 07:17:34 dist-test-slave-gjrz krb5kdc[3276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:34.908420 3296 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:34.909237 3296 webserver.cc:460] Webserver started at http://127.12.117.190:33089/ using document root <none> and password file <none>
I20260810 07:17:34.909720 3296 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:34.909783 3296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:34.909962 3296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:34.911870 3296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "4fc73f2d1ef54f539ec260683659f98b"
format_stamp: "Formatted at 2026-08-10 07:17:34 on dist-test-slave-gjrz"
server_key: "2f01c0c8fd861cf839c6aee034c60b5a"
server_key_iv: "3848805e8f34d3e3c572e47e5d204332"
server_key_version: "encryptionkey@0"
I20260810 07:17:34.912389 3296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "4fc73f2d1ef54f539ec260683659f98b"
format_stamp: "Formatted at 2026-08-10 07:17:34 on dist-test-slave-gjrz"
server_key: "2f01c0c8fd861cf839c6aee034c60b5a"
server_key_iv: "3848805e8f34d3e3c572e47e5d204332"
server_key_version: "encryptionkey@0"
I20260810 07:17:34.915751 3296 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:17:34.918022 3318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:34.919087 3296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:17:34.919199 3296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "4fc73f2d1ef54f539ec260683659f98b"
format_stamp: "Formatted at 2026-08-10 07:17:34 on dist-test-slave-gjrz"
server_key: "2f01c0c8fd861cf839c6aee034c60b5a"
server_key_iv: "3848805e8f34d3e3c572e47e5d204332"
server_key_version: "encryptionkey@0"
I20260810 07:17:34.919317 3296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:34.948748 3296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:34.951190 3296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:34.951363 3296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:34.957582 3296 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:44403
I20260810 07:17:34.957823 3398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:44403 every 8 connection(s)
I20260810 07:17:34.958425 3296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:34.961030 3399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:34.962234 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 3296
I20260810 07:17:34.962328 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:17:34.962590 12758 external_mini_cluster.cc:1428] Setting key 052beae2d7ac36d213ec84ca1eec2170
I20260810 07:17:34.964843 3399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.117.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.189@KRBTEST.COM" created.
Aug 10 07:17:34 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:34.979511 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.967249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44403 (local address 127.0.0.1:42270)
0810 07:17:34.967739 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.967788 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:34.968549 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:34.969321 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:34.969325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:34.969630 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:34.969883 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:34.969895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:34.971531 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:34.971533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:34.972259 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:34.972268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:34.972883 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:34.972886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:34.972975 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:34.973399 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:34.973413 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:34.975131 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:34.976887 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:34.976897 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:34.976906 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:34.977138 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:34.977481 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:34.977483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:34.977485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:34.977580 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:34.978265 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:34.978271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:34.978417 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 07:17:34.978549 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 07:17:34.978777 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260810 07:17:34.979514 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.967250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.12.117.190:44403)
0810 07:17:34.967809 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:34.967824 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:34.967869 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:17:34.968876 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:34.968882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:34.969131 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:34.969338 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:34.970027 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:34.971420 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:34.972367 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:34.972756 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:34.972874 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:34.975295 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:34.975303 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:34.975315 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:34.975337 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:34.976777 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:34.977251 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:34.977257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:34.977259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:34.977398 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:34.977662 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:34.977665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:34.977667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:34.978164 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:34.978307 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:34.978653 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 07:17:34.978880 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":427,"spinlock_wait_cycles":30592,"thread_start_us":167,"threads_started":1}
I20260810 07:17:34.981699 3399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:34.982344 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.982130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37126)
0810 07:17:34.982233 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.982294 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:17:34.982512 3326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
I20260810 07:17:34.983244 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:34.983042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37140)
0810 07:17:34.983150 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:34.983195 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:17:34.983392 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:42357: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
I20260810 07:17:34.988232 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:42357
--webserver_interface=127.12.117.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:35.032593 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } attempt: 1
I20260810 07:17:35.033352 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.033069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37142)
0810 07:17:35.033216 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.033275 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 07:17:35.034092 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.033889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37152)
0810 07:17:35.033988 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.034033 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:17:35.034211 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:42357: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
W20260810 07:17:35.094187 3408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:35.094394 3408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:35.094431 3408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:35.098294 3408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:35.098342 3408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:35.098361 3408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:35.098374 3408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:35.098390 3408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:35.102353 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } attempt: 2
I20260810 07:17:35.103135 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.102860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37154)
0810 07:17:35.102991 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.103052 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 07:17:35.103597 3408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:42357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.117.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:35.103827 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.103599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37160)
0810 07:17:35.103695 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.103751 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:17:35.103937 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:42357: Network error: Client connection negotiation failed: client connection to 127.12.117.189:42357: connect: Connection refused (error 111)
I20260810 07:17:35.104645 3408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:35.105507 3408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:35.113170 3418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:35.113164 3423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:35.113147 3419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:35.113253 3408 server_base.cc:1034] running on GCE node
I20260810 07:17:35.113869 3408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:35.114825 3408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:35.115971 3408 hybrid_clock.cc:648] HybridClock initialized: now 1786346255115953 us; error 29 us; skew 500 ppm
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:35.118454 3408 init.cc:377] Logged in from keytab as kudu/127.12.117.189@KRBTEST.COM (short username kudu)
I20260810 07:17:35.119232 3408 webserver.cc:460] Webserver started at http://127.12.117.189:39879/ using document root <none> and password file <none>
I20260810 07:17:35.119740 3408 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:35.119798 3408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:35.119980 3408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:35.121812 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data/instance:
uuid: "51dcac64fc134087ab0658d6f4ad4f72"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "2e86ef4c78b8f564b88cccafce1e6272"
server_key_iv: "3929c792cf8d6f99add48ad87184b6f9"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.122345 3408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal/instance:
uuid: "51dcac64fc134087ab0658d6f4ad4f72"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "2e86ef4c78b8f564b88cccafce1e6272"
server_key_iv: "3929c792cf8d6f99add48ad87184b6f9"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.125751 3408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260810 07:17:35.127918 3438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:35.129017 3408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:17:35.129132 3408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "51dcac64fc134087ab0658d6f4ad4f72"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "2e86ef4c78b8f564b88cccafce1e6272"
server_key_iv: "3929c792cf8d6f99add48ad87184b6f9"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.129216 3408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:35.141448 3408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:35.143862 3408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:35.144107 3408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:35.150597 3408 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.189:42357
I20260810 07:17:35.150848 3544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.189:42357 every 8 connection(s)
I20260810 07:17:35.151508 3408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
I20260810 07:17:35.153455 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 3408
I20260810 07:17:35.153560 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-1/wal/instance
I20260810 07:17:35.153816 12758 external_mini_cluster.cc:1428] Setting key 04acc5665292df4e92a6e685e4344858
I20260810 07:17:35.154227 3545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:35.157724 3545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.117.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.188@KRBTEST.COM" created.
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:35.171481 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.159696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.12.117.190:44403)
0810 07:17:35.159841 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.159846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.160637 (+ 791us) server_negotiation.cc:407] Connection header received
0810 07:17:35.161534 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.161538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.161598 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.161701 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.162812 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.163387 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.164537 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.164786 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.164880 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.167466 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.167470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.167472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.167498 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.169501 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.169975 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.169979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.169981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.170059 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.170373 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.170376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.170378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.170569 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.170658 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.171158 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.171317 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:17:35.172199 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.160028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44403 (local address 127.0.0.1:42286)
0810 07:17:35.160473 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.160523 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.161323 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.161833 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.161839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.162201 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.162663 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.162677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.163514 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.163517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.164419 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.164430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.164930 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.164935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.165109 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.165684 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.165706 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.167306 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.169615 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.169620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.169634 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.169878 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.170147 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.170150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.170152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.170283 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.170673 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.170680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.170911 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.171082 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.171420 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":109,"threads_started":1}
I20260810 07:17:35.173523 3545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
I20260810 07:17:35.177884 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:35291
--webserver_interface=127.12.117.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:17:35.182144 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.173943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37170)
0810 07:17:35.174049 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.174063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.174144 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.174727 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.174730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.174890 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.174975 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.174981 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.176020 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.176023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.176611 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.176621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.177197 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.177200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.177355 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.177741 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.177755 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.178907 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.180629 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.180632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.180634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.180866 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.181138 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.181141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.181143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.181240 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.181701 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.181704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.181787 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.181883 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.181998 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":51}
I20260810 07:17:35.182260 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.174052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.12.117.189:42357)
0810 07:17:35.174369 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.174377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.174395 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:17:35.174437 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.174441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.174599 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.174750 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.175081 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.175913 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.176711 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.177097 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.177187 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.179055 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.179063 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.179082 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.179103 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.180503 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.180953 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.180976 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.180978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.181058 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.181394 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.181397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.181398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.181606 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.181719 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.181967 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.182093 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
I20260810 07:17:35.183956 3545 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:35.184568 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.184350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48886)
0810 07:17:35.184449 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.184516 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:17:35.184769 3448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
I20260810 07:17:35.185586 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.185385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48890)
0810 07:17:35.185483 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.185524 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:17:35.185726 3545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
I20260810 07:17:35.195094 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } attempt: 3
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:17:35.202769 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.195444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37176)
0810 07:17:35.195610 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.195620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.195681 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.195880 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.195882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.196009 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.196112 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.196117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.197035 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.197037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.197562 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.197568 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.198128 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.198131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.198214 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.198591 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.198602 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.199876 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.201485 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.201487 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.201488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.201692 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.201956 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.201958 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.201959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.202023 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.202377 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.202379 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.202450 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.202537 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.202661 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 07:17:35.202838 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.195524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37176 (local address 127.12.117.189:42357)
0810 07:17:35.195630 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.195634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.195651 (+ 17us) server_negotiation.cc:407] Connection header received
0810 07:17:35.195756 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.195759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.195801 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.195870 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.196202 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.196942 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.197660 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.198012 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.198125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.200010 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.200014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.200017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.200042 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.201387 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.201776 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.201780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.201781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.201880 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.202091 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.202094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.202095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.202299 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.202395 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.202596 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.202706 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:17:35.204031 3399 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:35.204636 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.204428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48900)
0810 07:17:35.204530 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.204582 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:17:35.205250 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.205053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48908)
0810 07:17:35.205168 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.205206 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 07:17:35.205360 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
I20260810 07:17:35.234916 3545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 1
I20260810 07:17:35.235518 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.235300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48920)
0810 07:17:35.235406 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.235456 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 07:17:35.236155 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.235974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48924)
0810 07:17:35.236062 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.236103 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:17:35.236261 3545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
I20260810 07:17:35.236486 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 1
I20260810 07:17:35.237006 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.236815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48934)
0810 07:17:35.236912 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.236953 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:17:35.237553 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.237397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48940)
0810 07:17:35.237472 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.237510 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 07:17:35.237671 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
W20260810 07:17:35.291914 3556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:35.292135 3556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:35.292178 3556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:35.296077 3556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:35.296123 3556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:35.296142 3556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:35.296172 3556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:35.296197 3556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:35.301260 3556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44403,127.12.117.189:42357,127.12.117.188:35291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:35291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.117.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:35.302157 3556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:35.303073 3556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:35.304394 3545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 2
I20260810 07:17:35.305176 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.304863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48948)
0810 07:17:35.305005 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.305071 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 07:17:35.305879 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.305692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48960)
0810 07:17:35.305784 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.305825 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:17:35.305998 3545 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
W20260810 07:17:35.311105 3564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:35.311103 3563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:35.311218 3569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:35.311506 3556 server_base.cc:1034] running on GCE node
I20260810 07:17:35.312048 3556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:35.312788 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 2
I20260810 07:17:35.313042 3556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:35.313403 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.313140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48976)
0810 07:17:35.313266 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.313323 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:17:35.314064 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.313859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:48992)
0810 07:17:35.313958 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.314008 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:17:35.314172 3399 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35291: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35291: connect: Connection refused (error 111)
I20260810 07:17:35.314211 3556 hybrid_clock.cc:648] HybridClock initialized: now 1786346255314184 us; error 38 us; skew 500 ppm
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:35.316740 3556 init.cc:377] Logged in from keytab as kudu/127.12.117.188@KRBTEST.COM (short username kudu)
I20260810 07:17:35.317520 3556 webserver.cc:460] Webserver started at http://127.12.117.188:39759/ using document root <none> and password file <none>
I20260810 07:17:35.318003 3556 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:35.318060 3556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:35.318222 3556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:35.320113 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data/instance:
uuid: "1df7947d058240e686d56bb7ab788afc"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "28693484239da945c6834801730b9913"
server_key_iv: "42b8919f1c2ac7ac8bd7431c6e8b840b"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.320598 3556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal/instance:
uuid: "1df7947d058240e686d56bb7ab788afc"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "28693484239da945c6834801730b9913"
server_key_iv: "42b8919f1c2ac7ac8bd7431c6e8b840b"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.324120 3556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:17:35.326272 3583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:35.327409 3556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:17:35.327513 3556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "1df7947d058240e686d56bb7ab788afc"
format_stamp: "Formatted at 2026-08-10 07:17:35 on dist-test-slave-gjrz"
server_key: "28693484239da945c6834801730b9913"
server_key_iv: "42b8919f1c2ac7ac8bd7431c6e8b840b"
server_key_version: "encryptionkey@0"
I20260810 07:17:35.327615 3556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:35.340049 3556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:35.342480 3556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:35.342648 3556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:35.348984 3556 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.188:35291
I20260810 07:17:35.349231 3687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.188:35291 every 8 connection(s)
I20260810 07:17:35.349872 3556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
I20260810 07:17:35.352476 3689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:35.353348 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 3556
I20260810 07:17:35.353466 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786346097807571-12758-0/minicluster-data/master-2/wal/instance
I20260810 07:17:35.353765 12758 external_mini_cluster.cc:1428] Setting key 02431eae09b7836feca9622b5921b339
I20260810 07:17:35.356422 3689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:35.366340 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.358249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.12.117.190:44403)
0810 07:17:35.358351 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.358355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.358671 (+ 316us) server_negotiation.cc:407] Connection header received
0810 07:17:35.358803 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.358805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.358849 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.358912 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.359537 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.360010 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.360840 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.361039 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.361131 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.363458 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.363461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.363463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.363487 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.364697 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.365118 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.365121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.365123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.365195 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.365464 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.365468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.365469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.365649 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.365732 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.366065 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.366186 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:17:35.367080 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.354970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48994 (local address 127.12.117.188:35291)
0810 07:17:35.355379 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.355394 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.355589 (+ 195us) server_negotiation.cc:407] Connection header received
0810 07:17:35.356307 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.356323 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.356628 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.356932 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.357791 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.359144 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.359991 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.360458 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.360625 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.362992 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.362998 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.363011 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.363036 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.364674 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.365070 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.365075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.365076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.365180 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.365439 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.365443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.365444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.365777 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.365950 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.366174 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.366376 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260810 07:17:35.367081 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.358232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:44403 (local address 127.0.0.1:42294)
0810 07:17:35.358548 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.358593 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.358704 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.358969 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.358974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.359306 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.359426 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.359432 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.360118 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.360122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.360734 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.360742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.361164 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.361168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.361265 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.361726 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.361741 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.363326 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.364798 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.364807 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.364816 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.365021 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.365274 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.365277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.365279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.365380 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.365745 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.365751 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.365870 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.366007 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.366289 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":125,"threads_started":1}
I20260810 07:17:35.368225 3689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:35.375712 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.368632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:42357 (local address 127.0.0.1:37188)
0810 07:17:35.368709 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.368721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.368785 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.369070 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.369072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.369237 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.369351 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.369358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.370195 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.370199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.370714 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.370723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.371140 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.371143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.371239 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.371649 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.371660 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.372848 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.374354 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.374356 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.374358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.374598 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.374838 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.374841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.374842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.374917 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.375259 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.375290 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.375369 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.375451 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.375565 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260810 07:17:35.375751 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.368741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.12.117.189:42357)
0810 07:17:35.368866 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.368869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.368880 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:17:35.368918 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.368921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.368964 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.369040 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.369461 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.370085 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.370828 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.371032 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.371109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.372971 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.372974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.372976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.372998 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.374255 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.374676 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.374680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.374682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.374757 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.374995 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.374998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.374999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.375173 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.375254 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.375515 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.375619 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:17:35.376356 3689 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:17:35.377367 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.369351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.12.117.190:44403)
0810 07:17:35.369441 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.369445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.369583 (+ 138us) server_negotiation.cc:407] Connection header received
0810 07:17:35.369649 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.369652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.369691 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.369764 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.370258 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.370697 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.371562 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.371803 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.371878 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.374398 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.374402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.374404 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.374426 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.375714 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.376209 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.376212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.376213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.376300 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.376559 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.376562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.376563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.376767 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.376960 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.377125 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.377225 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:17:35.384276 3696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.376762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:49000)
0810 07:17:35.377033 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.377043 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.377114 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.377350 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.377352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.377488 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.377560 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.377567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.378747 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.378750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.379235 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.379242 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.379693 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.379696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.379783 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.380151 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.380164 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.381177 (+ 1013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.382932 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.382935 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.382937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.383175 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.383433 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.383436 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.383438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.383510 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.383824 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.383827 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.383904 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.383984 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.384118 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1}
I20260810 07:17:35.384294 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.376827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.12.117.188:35291)
0810 07:17:35.376919 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.376923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.377103 (+ 180us) server_negotiation.cc:407] Connection header received
0810 07:17:35.377176 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.377178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.377211 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.377268 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.377662 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.378650 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.379365 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.379595 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.379685 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.381304 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.381308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.381311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.381335 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.382484 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.383280 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.383284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.383285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.383357 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.383589 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.383592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.383594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.383742 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.383821 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.384035 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.384174 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":40}
I20260810 07:17:35.385964 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.378416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.12.117.189:42357)
0810 07:17:35.378507 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.378511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.378523 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:17:35.378614 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.378618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.378653 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.378715 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.379237 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.379736 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.380413 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.380626 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.380705 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.382944 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.382948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.382950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.382972 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.384230 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.384841 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.384844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.384846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.384931 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.385253 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.385256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.385257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.385411 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.385635 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.385740 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.385835 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:17:35.387244 3689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: Bootstrap starting.
I20260810 07:17:35.388944 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:17:35.389648 3689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:35.390426 3689 log.cc:824] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:35.393319 3689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: No bootstrap required, opened a new log
I20260810 07:17:35.397125 3545 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 3
I20260810 07:17:35.397270 3689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.397503 3689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:35.397554 3689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1df7947d058240e686d56bb7ab788afc, State: Initialized, Role: FOLLOWER
I20260810 07:17:35.398057 3689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.399032 3712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.399232 3712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.399322 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.390721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.12.117.190:44403)
0810 07:17:35.390814 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.390818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.391035 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:17:35.391139 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.391141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.391187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.391284 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.392072 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.392762 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.393837 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.394150 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.394217 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.396036 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.396040 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.396043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.396070 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.397462 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.397934 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.397937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.397939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.398032 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.398362 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.398365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.398367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.398580 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.398694 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.399046 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.399153 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:17:35.399400 3689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:35.402012 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.391078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.12.117.189:42357)
0810 07:17:35.391173 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.391175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.391551 (+ 376us) server_negotiation.cc:407] Connection header received
0810 07:17:35.391656 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.391659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.391701 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.391777 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.392533 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.393204 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.394206 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.394494 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.394825 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.397093 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.397096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.397352 (+ 256us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.397378 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.399478 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.399914 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.399918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.399919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.400244 (+ 325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.400786 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.400790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.400792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.401000 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.401161 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.401755 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.401859 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:17:35.403290 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.391393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.12.117.188:35291)
0810 07:17:35.391482 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.391486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.392327 (+ 841us) server_negotiation.cc:407] Connection header received
0810 07:17:35.392361 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.392363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.392400 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.392458 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.393254 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.393965 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.395010 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.395350 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.395436 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.396893 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.396897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.396900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.399803 (+ 2903us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.401263 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.401729 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.401732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.401734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.401814 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.402147 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.402150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.402152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.402369 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.402476 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.403040 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.403141 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 07:17:35.404985 3727 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:35.405036 3727 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:35.402905 3556 master_runner.cc:186] Error getting master registration for 127.12.117.190:44403: OK, instance_id { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" instance_seqno: 1786346254955910 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:17:35.405990 3556 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346255, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:17:35.410497 3551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.397489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:49018)
0810 07:17:35.397644 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.397658 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.398023 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.398930 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.398933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.399145 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.399236 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.399243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.404272 (+ 5029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.404276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.405050 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.405061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.405641 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.405644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.405706 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.406071 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.406090 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.407109 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.409125 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.409128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.409130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.409368 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.409720 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.409722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.409723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.409799 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.410172 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.410174 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.410242 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.410312 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.410402 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 07:17:35.410796 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.398192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.12.117.188:35291)
0810 07:17:35.398706 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.398716 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.398728 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:17:35.398760 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.398763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.398800 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.403226 (+ 4426us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.403365 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.404143 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.405210 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.405533 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.405634 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.407235 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.407238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.407241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.407300 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.408996 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.409494 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.409497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.409499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.409611 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.409881 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.409884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.409886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.410083 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.410209 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.410379 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.410489 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":302,"threads_started":1}
I20260810 07:17:35.413085 3545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: Bootstrap starting.
I20260810 07:17:35.413324 3399 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } attempt: 3
I20260810 07:17:35.415661 3545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:35.417032 3545 log.cc:824] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:17:35 dist-test-slave-gjrz krb5kdc[3276](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346254, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:17:35.419014 3545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: No bootstrap required, opened a new log
I20260810 07:17:35.422408 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.413687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35291 (local address 127.0.0.1:49034)
0810 07:17:35.413847 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:35.413860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:35.413944 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:35.414175 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:35.414178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:35.414347 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:35.414439 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.414445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.415402 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.415406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.416232 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:35.416245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.417125 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.417130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:35.417217 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.417651 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:35.417665 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:35.419028 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:35.420894 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.420897 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.420899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.421125 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.421411 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:35.421414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:35.421416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:35.421496 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:35.421950 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:35.421953 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:35.422040 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 07:17:35.422140 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:17:35.422257 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 07:17:35.422459 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:35.413750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.12.117.188:35291)
0810 07:17:35.413852 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:35.413856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:35.413906 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:17:35.414026 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:35.414029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:35.414067 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:35.414148 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:35.414564 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.415286 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.416643 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:35.416992 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:35.417100 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:35.419174 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:35.419177 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:35.419180 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:35.419202 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:35.420774 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.421233 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.421237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.421239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.421318 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:35.421601 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:35.421604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:35.421606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:35.421803 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:35.421904 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:35.422217 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 07:17:35.422326 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:17:35.422884 3545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.423106 3545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:35.423166 3545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51dcac64fc134087ab0658d6f4ad4f72, State: Initialized, Role: FOLLOWER
I20260810 07:17:35.423612 3545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.424067 3731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.424252 3731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.424834 3545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:35.425305 3399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: Bootstrap starting.
I20260810 07:17:35.427860 3399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:35.428686 3399 log.cc:824] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: Log is configured to *not* fsync() on all Append() calls
W20260810 07:17:35.431059 3748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:35.431149 3748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:35.428877 3408 master_runner.cc:186] Error getting master registration for 127.12.117.190:44403: OK, instance_id { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" instance_seqno: 1786346254955910 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:17:35.432132 3408 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:17:35.433544 3399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: No bootstrap required, opened a new log
I20260810 07:17:35.436391 3399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.436564 3399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:35.436625 3399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc73f2d1ef54f539ec260683659f98b, State: Initialized, Role: FOLLOWER
I20260810 07:17:35.437000 3399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.437405 3755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.437575 3755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.438081 3399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 07:17:35.444012 3774 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:17:35.444147 3774 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:17:35.446085 3296 master_runner.cc:263] No leader master found from master 4fc73f2d1ef54f539ec260683659f98b
I20260810 07:17:35.446152 3296 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:17:35.452754 3329 tablet.cc:2369] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:17:35.593840 3592 tablet.cc:2369] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:17:35.645581 3454 tablet.cc:2369] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:35.842600 3712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:35.842806 3712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.843724 3712 leader_election.cc:288] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fc73f2d1ef54f539ec260683659f98b (127.12.117.190:44403), 51dcac64fc134087ab0658d6f4ad4f72 (127.12.117.189:42357)
I20260810 07:17:35.844198 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1df7947d058240e686d56bb7ab788afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc73f2d1ef54f539ec260683659f98b" is_pre_election: true
I20260810 07:17:35.844225 3488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1df7947d058240e686d56bb7ab788afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51dcac64fc134087ab0658d6f4ad4f72" is_pre_election: true
I20260810 07:17:35.844501 3488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1df7947d058240e686d56bb7ab788afc in term 0.
I20260810 07:17:35.844512 3362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1df7947d058240e686d56bb7ab788afc in term 0.
I20260810 07:17:35.844907 3590 leader_election.cc:302] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1df7947d058240e686d56bb7ab788afc, 51dcac64fc134087ab0658d6f4ad4f72; no voters:
I20260810 07:17:35.845148 3712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:35.845234 3712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:35.845284 3712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:35.846313 3712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.846589 3712 leader_election.cc:288] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [CANDIDATE]: Term 1 election: Requested vote from peers 4fc73f2d1ef54f539ec260683659f98b (127.12.117.190:44403), 51dcac64fc134087ab0658d6f4ad4f72 (127.12.117.189:42357)
I20260810 07:17:35.846903 3362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1df7947d058240e686d56bb7ab788afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc73f2d1ef54f539ec260683659f98b"
I20260810 07:17:35.846936 3488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1df7947d058240e686d56bb7ab788afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51dcac64fc134087ab0658d6f4ad4f72"
I20260810 07:17:35.847015 3362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:35.847033 3488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:35.848204 3488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1df7947d058240e686d56bb7ab788afc in term 1.
I20260810 07:17:35.848371 3362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1df7947d058240e686d56bb7ab788afc in term 1.
I20260810 07:17:35.848506 3590 leader_election.cc:302] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1df7947d058240e686d56bb7ab788afc, 51dcac64fc134087ab0658d6f4ad4f72; no voters:
I20260810 07:17:35.848827 3712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:35.849009 3712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [term 1 LEADER]: Becoming Leader. State: Replica: 1df7947d058240e686d56bb7ab788afc, State: Running, Role: LEADER
I20260810 07:17:35.849280 3712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } }
I20260810 07:17:35.850965 3776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1df7947d058240e686d56bb7ab788afc. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.851063 3776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:35.851409 3780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:35.852218 3780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:35.856484 3362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [term 1 FOLLOWER]: Refusing update from remote peer 1df7947d058240e686d56bb7ab788afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:35.856484 3488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [term 1 FOLLOWER]: Refusing update from remote peer 1df7947d058240e686d56bb7ab788afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:35.857059 3712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:35.857231 3776 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:35.861095 3731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1df7947d058240e686d56bb7ab788afc. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.861099 3755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1df7947d058240e686d56bb7ab788afc. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.861208 3731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.861208 3755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.863814 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.863935 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:35.864049 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.864058 3755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.864183 3755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.864174 3731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1df7947d058240e686d56bb7ab788afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc73f2d1ef54f539ec260683659f98b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44403 } } peers { permanent_uuid: "51dcac64fc134087ab0658d6f4ad4f72" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 42357 } } peers { permanent_uuid: "1df7947d058240e686d56bb7ab788afc" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35291 } } }
I20260810 07:17:35.864277 3731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:17:35.864255 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:35.864389 3785 mvcc.cc:205] Tried to move back new op lower bound from 7316874263982030848 to 7316874263960993792. Current Snapshot: MvccSnapshot[applied={T|T < 7316874263982030848}]
I20260810 07:17:35.865679 3780 catalog_manager.cc:1347] Generated new cluster ID: 50521a32d72f4e73a6d4efca2c0d1097
I20260810 07:17:35.865741 3780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:35.873934 3780 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:35.875022 3780 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:35.882344 3780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1df7947d058240e686d56bb7ab788afc: Generated new TSK 0
I20260810 07:17:35.883033 3780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:17:36.432382 3748 catalog_manager.cc:1259] Loaded cluster ID: 50521a32d72f4e73a6d4efca2c0d1097
I20260810 07:17:36.432471 3748 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: loading cluster ID for follower catalog manager: success
I20260810 07:17:36.434192 3748 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: acquiring CA information for follower catalog manager: success
I20260810 07:17:36.435365 3748 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 51dcac64fc134087ab0658d6f4ad4f72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:17:36.445205 3774 catalog_manager.cc:1259] Loaded cluster ID: 50521a32d72f4e73a6d4efca2c0d1097
I20260810 07:17:36.445266 3774 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: loading cluster ID for follower catalog manager: success
I20260810 07:17:36.447206 3774 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: acquiring CA information for follower catalog manager: success
I20260810 07:17:36.448498 3774 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4fc73f2d1ef54f539ec260683659f98b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:17:36.962487 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 3296
I20260810 07:17:36.979065 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 3408
I20260810 07:17:36.993156 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 3556
2026-08-10T07:17:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3501 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:37 dist-test-slave-gjrz krb5kdc[3805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:37 dist-test-slave-gjrz krb5kdc[3805](info): set up 2 sockets
Aug 10 07:17:37 dist-test-slave-gjrz krb5kdc[3805](info): commencing operation
krb5kdc: starting...
W20260810 07:17:38.159909 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:17:38.275766 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37229
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:38.384501 3837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:38.384745 3837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:38.384801 3837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:38.388732 3837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:38.388785 3837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:38.388811 3837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:38.388833 3837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:38.388860 3837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:38.393954 3837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:38.394873 3837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:38.395836 3837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:38.403851 3845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:38.403846 3849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:38.403905 3837 server_base.cc:1034] running on GCE node
W20260810 07:17:38.403846 3846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:38.404593 3837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:38.405633 3837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:38.406783 3837 hybrid_clock.cc:648] HybridClock initialized: now 1786346258406762 us; error 40 us; skew 500 ppm
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346258, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:38.409313 3837 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:38.410120 3837 webserver.cc:460] Webserver started at http://127.12.117.190:41897/ using document root <none> and password file <none>
I20260810 07:17:38.410648 3837 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:38.410717 3837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:38.410895 3837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:38.412863 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "922a098e95844aadbca0f1bb1c9927b9"
format_stamp: "Formatted at 2026-08-10 07:17:38 on dist-test-slave-gjrz"
server_key: "6416b6a30c152b5e5a435df105e06a1b"
server_key_iv: "a2dee56009074bfc4da7614abed679c4"
server_key_version: "encryptionkey@0"
I20260810 07:17:38.413375 3837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "922a098e95844aadbca0f1bb1c9927b9"
format_stamp: "Formatted at 2026-08-10 07:17:38 on dist-test-slave-gjrz"
server_key: "6416b6a30c152b5e5a435df105e06a1b"
server_key_iv: "a2dee56009074bfc4da7614abed679c4"
server_key_version: "encryptionkey@0"
I20260810 07:17:38.416826 3837 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 07:17:38.419030 3860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:38.420164 3837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:17:38.420275 3837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "922a098e95844aadbca0f1bb1c9927b9"
format_stamp: "Formatted at 2026-08-10 07:17:38 on dist-test-slave-gjrz"
server_key: "6416b6a30c152b5e5a435df105e06a1b"
server_key_iv: "a2dee56009074bfc4da7614abed679c4"
server_key_version: "encryptionkey@0"
I20260810 07:17:38.420365 3837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:38.450078 3837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:38.452795 3837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:38.453034 3837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:38.459703 3837 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37229
I20260810 07:17:38.459738 3951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37229 every 8 connection(s)
I20260810 07:17:38.460593 3837 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:38.461045 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 3837
I20260810 07:17:38.461179 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:17:38.461447 12758 external_mini_cluster.cc:1428] Setting key 4e3c9c89263f0174706977db2fca4031
I20260810 07:17:38.463549 3952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:38.469759 3952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9: Bootstrap starting.
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:38.471872 3952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:38.472617 3952 log.cc:824] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:38.474328 3952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9: No bootstrap required, opened a new log
I20260810 07:17:38.475070 3956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:38.462708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.12.117.190:37229)
0810 07:17:38.463111 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:38.463127 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:38.463163 (+ 36us) server_negotiation.cc:407] Connection header received
0810 07:17:38.463849 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:38.463869 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:38.464187 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:38.464456 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:38.465326 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:38.466315 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:38.467469 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:38.467929 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:38.468042 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:38.470679 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:38.470684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:38.470698 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:38.470725 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:38.472335 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:38.472844 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:38.472848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:38.472853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:38.472959 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:38.473245 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:38.473249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:38.473250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:38.473630 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:38.473816 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:38.474093 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 07:17:38.474364 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":120,"threads_started":1}
I20260810 07:17:38.477885 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:17:38.478070 3952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } }
I20260810 07:17:38.478322 3952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:38.478377 3952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922a098e95844aadbca0f1bb1c9927b9, State: Initialized, Role: FOLLOWER
I20260810 07:17:38.478808 3952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } }
I20260810 07:17:38.478935 3952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:38.479019 3952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:38.479126 3952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:38.479986 3952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } }
I20260810 07:17:38.480291 3952 leader_election.cc:302] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922a098e95844aadbca0f1bb1c9927b9; no voters:
I20260810 07:17:38.480545 3952 leader_election.cc:288] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:38.480686 3958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:38.480913 3958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [term 1 LEADER]: Becoming Leader. State: Replica: 922a098e95844aadbca0f1bb1c9927b9, State: Running, Role: LEADER
I20260810 07:17:38.481197 3958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } }
I20260810 07:17:38.481619 3952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:38.482685 3962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922a098e95844aadbca0f1bb1c9927b9. Latest consensus state: current_term: 1 leader_uuid: "922a098e95844aadbca0f1bb1c9927b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } } }
I20260810 07:17:38.482821 3962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:38.483197 3971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:38.483165 3960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922a098e95844aadbca0f1bb1c9927b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922a098e95844aadbca0f1bb1c9927b9" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37229 } } }
I20260810 07:17:38.483349 3960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:38.485882 3971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:38.490484 3971 catalog_manager.cc:1347] Generated new cluster ID: a7c2ac8c8c1144ecad544aafe57a1e94
I20260810 07:17:38.490546 3971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:38.526624 3971 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:38.527518 3971 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:38.534313 3971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922a098e95844aadbca0f1bb1c9927b9: Generated new TSK 0
I20260810 07:17:38.534885 3971 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:17:38.584542 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 3837
2026-08-10T07:17:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1596 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3997](info): set up 2 sockets
Aug 10 07:17:38 dist-test-slave-gjrz krb5kdc[3997](info): commencing operation
krb5kdc: starting...
W20260810 07:17:39.764552 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:17:39 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:17:39.875092 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:39.979065 4018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:39.979317 4018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:39.979377 4018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:39.983282 4018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:39.983332 4018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:39.983359 4018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:39.983381 4018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:39.983405 4018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:39.988401 4018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:39.989298 4018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:39.990221 4018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:39.998073 4027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:39.998149 4018 server_base.cc:1034] running on GCE node
W20260810 07:17:39.998116 4026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:39.998136 4030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:39.998857 4018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:39.999854 4018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:40.001013 4018 hybrid_clock.cc:648] HybridClock initialized: now 1786346260000995 us; error 31 us; skew 500 ppm
Aug 10 07:17:39 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346259, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:40.003563 4018 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:40.004382 4018 webserver.cc:460] Webserver started at http://127.12.117.190:39033/ using document root <none> and password file <none>
I20260810 07:17:40.004874 4018 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:40.004928 4018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:40.005095 4018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:40.006987 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "11abe7d525dc431fa5e70f480de55c90"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "de2f3691bdd3924c699d0069bfd01825"
server_key_iv: "db87e5904ec75e6055a8a62f38cf30ce"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.007542 4018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "11abe7d525dc431fa5e70f480de55c90"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "de2f3691bdd3924c699d0069bfd01825"
server_key_iv: "db87e5904ec75e6055a8a62f38cf30ce"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.010953 4018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260810 07:17:40.013195 4044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.014304 4018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:40.014398 4018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "11abe7d525dc431fa5e70f480de55c90"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "de2f3691bdd3924c699d0069bfd01825"
server_key_iv: "db87e5904ec75e6055a8a62f38cf30ce"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.014477 4018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:40.025302 4018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:40.027747 4018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:40.027907 4018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:40.034323 4018 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39829
I20260810 07:17:40.034571 4132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39829 every 8 connection(s)
I20260810 07:17:40.035198 4018 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:40.037827 4134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:40.039707 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 4018
I20260810 07:17:40.039805 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:17:40.040051 12758 external_mini_cluster.cc:1428] Setting key f4051cbb97f9b86643b72a4395fa320f
I20260810 07:17:40.044375 4134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap starting.
I20260810 07:17:40.046891 4134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:40.047752 4134 log.cc:824] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:40.049883 4134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: No bootstrap required, opened a new log
I20260810 07:17:40.053623 4134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:40.053782 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.041308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.12.117.190:39829)
0810 07:17:40.041724 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.041743 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.041786 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:17:40.042485 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.042508 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.042922 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.043311 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:40.043914 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.045321 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.046164 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.046847 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.047026 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.049644 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:40.049649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:40.049661 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:40.049685 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:40.051192 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.051693 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.051697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.051702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.051805 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.052076 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.052079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.052080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.052421 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:40.052567 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.052840 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.053097 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":105,"threads_started":1}
I20260810 07:17:40.053844 4134 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:40.053896 4134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Initialized, Role: FOLLOWER
I20260810 07:17:40.054298 4134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:40.054416 4134 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:40.054476 4134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:40.054576 4134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:40.055416 4134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:40.055738 4134 leader_election.cc:302] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11abe7d525dc431fa5e70f480de55c90; no voters:
I20260810 07:17:40.056035 4134 leader_election.cc:288] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:40.056120 4141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:40.056332 4141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 LEADER]: Becoming Leader. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Running, Role: LEADER
I20260810 07:17:40.056643 4141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:40.057129 4134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:40.058086 4143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11abe7d525dc431fa5e70f480de55c90. Latest consensus state: current_term: 1 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:40.058223 4143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:40.058542 4142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:40.058600 4155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:40.058676 4142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:40.061719 4155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:40.066344 4155 catalog_manager.cc:1347] Generated new cluster ID: 1997d02d1c004252a74c18c8fdfc01f3
I20260810 07:17:40.066402 4155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:40.082829 4155 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:40.083724 4155 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:40.091907 4155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Generated new TSK 0
I20260810 07:17:40.092481 4155 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:17:40.123926 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:40.231362 4172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:40.231581 4172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:40.231637 4172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:40.235453 4172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:40.235499 4172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:40.235574 4172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:40.240424 4172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:40.241282 4172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:40.242174 4172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:40.250967 4172 server_base.cc:1034] running on GCE node
W20260810 07:17:40.250916 4180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.250978 4184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.250921 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:40.251511 4172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:40.252060 4172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:40.253242 4172 hybrid_clock.cc:648] HybridClock initialized: now 1786346260253216 us; error 46 us; skew 500 ppm
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:40.255743 4172 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:40.256526 4172 webserver.cc:460] Webserver started at http://127.12.117.129:45723/ using document root <none> and password file <none>
I20260810 07:17:40.257009 4172 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:40.257067 4172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:40.257274 4172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:40.259299 4172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "6769c5a05d414bbc97f1c7c468e25f20"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "83ff3a6b5a6e42bc4ff28b2c051e09b8"
server_key_iv: "0543e7b60415bb50a39daec4f544f3ff"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.259835 4172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "6769c5a05d414bbc97f1c7c468e25f20"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "83ff3a6b5a6e42bc4ff28b2c051e09b8"
server_key_iv: "0543e7b60415bb50a39daec4f544f3ff"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.263186 4172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:40.265450 4197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.266520 4172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:40.266613 4172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "6769c5a05d414bbc97f1c7c468e25f20"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "83ff3a6b5a6e42bc4ff28b2c051e09b8"
server_key_iv: "0543e7b60415bb50a39daec4f544f3ff"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.266707 4172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:40.276420 4172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:40.280215 4172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:40.280392 4172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:40.281052 4172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:40.282035 4172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:40.282097 4172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.282152 4172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:40.282181 4172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.293740 4172 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38139
I20260810 07:17:40.293793 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38139 every 8 connection(s)
I20260810 07:17:40.294514 4172 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:40.299539 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 4172
I20260810 07:17:40.299664 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:40.299944 12758 external_mini_cluster.cc:1428] Setting key a9d510417044689665d8a1062f342392
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:40.308388 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.297118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:36917 (local address 127.12.117.190:39829)
0810 07:17:40.297242 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.297246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.298087 (+ 841us) server_negotiation.cc:407] Connection header received
0810 07:17:40.299414 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.299418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.299475 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.299571 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:40.300786 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.301367 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.302499 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.302738 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.302827 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.305184 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:40.305188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:40.305191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:40.305220 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:40.306484 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.306949 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.306953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.306954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.307032 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.307361 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.307364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.307366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.307544 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:40.307642 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.308107 (+ 465us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.308232 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:17:40.309078 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.297515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:36917)
0810 07:17:40.297946 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.297987 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.299189 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.299740 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.299749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.300119 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:40.300648 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.300665 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.301491 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.301494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.302380 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.302390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.302872 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.302876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.303029 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.303559 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:40.303575 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:40.305028 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:40.306596 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.306602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.306616 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.306854 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.307122 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.307125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.307127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.307256 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.307645 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:40.307651 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:40.307874 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.308027 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.308329 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":103,"threads_started":1}
I20260810 07:17:40.310200 4393 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:40.310429 4393 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:40.310855 4393 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:17:40.312343 4072 ts_manager.cc:194] Registered new tserver with Master: 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:40.313704 4072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:36917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:17:40.326076 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:40.430944 4398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:40.431172 4398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:40.431210 4398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:40.435020 4398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:40.435066 4398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:40.435118 4398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:40.440003 4398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:40.440848 4398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:40.441736 4398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:40.450593 4412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.450603 4414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.450587 4419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:40.450906 4398 server_base.cc:1034] running on GCE node
I20260810 07:17:40.451251 4398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:40.451838 4398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:40.452989 4398 hybrid_clock.cc:648] HybridClock initialized: now 1786346260452974 us; error 26 us; skew 500 ppm
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:40.455482 4398 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:40.456234 4398 webserver.cc:460] Webserver started at http://127.12.117.130:38465/ using document root <none> and password file <none>
I20260810 07:17:40.456745 4398 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:40.456808 4398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:40.456986 4398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:40.458971 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "27aace958e024aec95a40bc13b35fc4f"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "4d5f0b5cf8d0e53d6bd6f615c7ebf6d1"
server_key_iv: "c45e74e16ac70eb34bd3c0c73a54fba5"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.459532 4398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "27aace958e024aec95a40bc13b35fc4f"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "4d5f0b5cf8d0e53d6bd6f615c7ebf6d1"
server_key_iv: "c45e74e16ac70eb34bd3c0c73a54fba5"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.462960 4398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 07:17:40.465217 4429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.466329 4398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 07:17:40.466423 4398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "27aace958e024aec95a40bc13b35fc4f"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "4d5f0b5cf8d0e53d6bd6f615c7ebf6d1"
server_key_iv: "c45e74e16ac70eb34bd3c0c73a54fba5"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.466502 4398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:40.480564 4398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:40.483377 4398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:40.483516 4398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:40.483987 4398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:40.484717 4398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:40.484764 4398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.484800 4398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:40.484820 4398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.492686 4398 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44393
I20260810 07:17:40.492743 4622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44393 every 8 connection(s)
I20260810 07:17:40.493489 4398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:40.501251 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 4398
I20260810 07:17:40.501369 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:40.501675 12758 external_mini_cluster.cc:1428] Setting key 67752176d2facf1741fcdc3fedc1dcfb
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:40.505872 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.495069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:37267 (local address 127.12.117.190:39829)
0810 07:17:40.495196 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.495200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.495965 (+ 765us) server_negotiation.cc:407] Connection header received
0810 07:17:40.497220 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.497222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.497272 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.497342 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:40.498413 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.498950 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.500040 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.500260 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.500335 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.502665 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:40.502669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:40.502671 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:40.502695 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:40.503986 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.504460 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.504464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.504466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.504544 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.504864 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.504868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.504870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.505047 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:40.505142 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.505596 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.505717 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:17:40.506580 4627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.495408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:37267)
0810 07:17:40.495818 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.495858 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.497038 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.497478 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.497484 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.497841 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:40.498273 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.498286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.499041 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.499043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.499947 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.499958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.500367 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.500369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.500496 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.500936 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:40.500951 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:40.502496 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:40.504100 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.504105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.504119 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.504362 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.504636 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.504639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.504641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.504772 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.505148 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:40.505154 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:40.505373 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.505534 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.505844 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":114,"threads_started":1}
I20260810 07:17:40.507659 4624 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:40.507876 4624 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:40.508340 4624 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
I20260810 07:17:40.509341 4072 ts_manager.cc:194] Registered new tserver with Master: 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
I20260810 07:17:40.510070 4072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:37267
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:17:40.527349 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:40.633769 4632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:40.633991 4632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:40.634044 4632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:40.637883 4632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:40.637934 4632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:40.638007 4632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:17:40.642936 4632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:40.643847 4632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:40.644748 4632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:40.653779 4647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.653785 4643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:40.653779 4641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:40.654083 4632 server_base.cc:1034] running on GCE node
I20260810 07:17:40.654448 4632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:40.655012 4632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:40.656163 4632 hybrid_clock.cc:648] HybridClock initialized: now 1786346260656135 us; error 38 us; skew 500 ppm
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:40.658779 4632 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:17:40.659783 4632 webserver.cc:460] Webserver started at http://127.12.117.131:33369/ using document root <none> and password file <none>
I20260810 07:17:40.660339 4632 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:40.660422 4632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:40.660669 4632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:40.662806 4632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "696bd1c5301e4638a086fac39cf822d0"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "1be580634f709c709edbd740b5bfb758"
server_key_iv: "77107937d5520bd9875c7da5645e51ca"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.663324 4632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "696bd1c5301e4638a086fac39cf822d0"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "1be580634f709c709edbd740b5bfb758"
server_key_iv: "77107937d5520bd9875c7da5645e51ca"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.666641 4632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:40.668795 4657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.669978 4632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:17:40.670089 4632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "696bd1c5301e4638a086fac39cf822d0"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "1be580634f709c709edbd740b5bfb758"
server_key_iv: "77107937d5520bd9875c7da5645e51ca"
server_key_version: "encryptionkey@0"
I20260810 07:17:40.670171 4632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:40.684051 4632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:40.686772 4632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:40.686906 4632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:40.687394 4632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:40.688112 4632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:40.688160 4632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.688195 4632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:40.688215 4632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:40.695981 4632 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:39919
I20260810 07:17:40.696009 4857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:39919 every 8 connection(s)
I20260810 07:17:40.696729 4632 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:40.702307 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 4632
I20260810 07:17:40.702400 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:17:40.702620 12758 external_mini_cluster.cc:1428] Setting key 31cfaa49655ab65ab4f1fd6a9f959d72
Aug 10 07:17:40 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346260, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:40.709019 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.698273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:41267 (local address 127.12.117.190:39829)
0810 07:17:40.698416 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.698419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.699103 (+ 684us) server_negotiation.cc:407] Connection header received
0810 07:17:40.700443 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.700446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.700507 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.700579 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:40.701648 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.702177 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.703460 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.703701 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.703796 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.705908 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:40.705911 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:40.705913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:40.705937 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:40.707129 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.707627 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.707630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.707631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.707707 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.708021 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.708023 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.708024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.708195 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:40.708279 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.708751 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.708868 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:17:40.709667 4861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.698553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:41267)
0810 07:17:40.698968 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.699006 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.700246 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.700758 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.700765 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.701076 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:40.701502 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.701515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.702297 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.702301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.703343 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.703354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.703836 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.703840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.704008 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.704467 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:40.704483 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:40.705759 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:40.707239 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.707245 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.707258 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.707525 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.707791 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:40.707795 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:40.707797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:40.707923 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:40.708297 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:40.708304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:40.708515 (+ 211us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.708672 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.708957 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":104,"threads_started":1}
I20260810 07:17:40.710565 4859 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:40.710765 4859 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:40.711167 4859 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
I20260810 07:17:40.712044 4072 ts_manager.cc:194] Registered new tserver with Master: 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919)
I20260810 07:17:40.712551 4072 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:41267
I20260810 07:17:40.715860 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:40.723549 4138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.717600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38548 (local address 127.12.117.190:39829)
0810 07:17:40.717702 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.717705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.717873 (+ 168us) server_negotiation.cc:407] Connection header received
0810 07:17:40.717961 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.717963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.717995 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.718051 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:40.718646 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.719122 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.719899 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.720096 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.720167 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.721186 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:40.721188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:40.721190 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:40.721211 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:40.722395 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.722695 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.722697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.722698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.722760 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:40.722924 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:40.722926 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:40.722927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:40.723095 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:40.723188 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.723363 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.723448 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:17:40.725754 4072 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:40.728516 4072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:40.741405 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.736080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.12.117.131:39919)
0810 07:17:40.736655 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.736666 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.736687 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:17:40.737258 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.737264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.737452 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.737794 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:40.737930 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.738628 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.739956 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.740756 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.740898 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.741025 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.741184 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.741270 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":136,"threads_started":1}
I20260810 07:17:40.741393 4882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.735968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:39919 (local address 127.0.0.1:47522)
0810 07:17:40.736294 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.736342 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.736460 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.737632 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.737638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.737658 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:40.737838 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.737848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.738744 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.738747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.739818 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.739829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.740903 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.740906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.741013 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.741049 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.741147 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.741263 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":44,"spinlock_wait_cycles":8192,"thread_start_us":77,"threads_started":1}
I20260810 07:17:40.742274 4887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.736802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.0.0.1:55576)
0810 07:17:40.737160 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.737173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.737254 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.737920 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.737923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.737940 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:40.738035 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.738041 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.739546 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.739549 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.740700 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.740725 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.741825 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.741829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.741985 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.742009 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.742089 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.742153 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":65,"thread_start_us":98,"threads_started":1}
I20260810 07:17:40.742360 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.736959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.12.117.129:38139)
0810 07:17:40.737492 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.737503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.737525 (+ 22us) server_negotiation.cc:407] Connection header received
0810 07:17:40.737599 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.737604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.737769 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.738188 (+ 419us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:40.738341 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.739430 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.740849 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.741670 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.741837 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.741985 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.742127 (+ 142us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.742214 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":120,"threads_started":1}
I20260810 07:17:40.742458 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.736141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44393 (local address 127.0.0.1:43724)
0810 07:17:40.736693 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.736839 (+ 146us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.736928 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.737644 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.737647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.737678 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:40.737808 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.737815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.739411 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.739414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.740662 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.740673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.741912 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.741916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.742047 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.742069 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.742143 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.742213 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":24,"spinlock_wait_cycles":2688,"thread_start_us":90,"threads_started":1}
I20260810 07:17:40.743767 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.736710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.12.117.130:44393)
0810 07:17:40.737176 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.737187 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.737212 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:17:40.737286 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.737291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.737480 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.737952 (+ 472us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:40.738114 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.739251 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.740814 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.741774 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.742914 (+ 1140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.743480 (+ 566us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.743533 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.743622 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":136,"threads_started":1}
I20260810 07:17:40.744364 4739 tablet_service.cc:1467] Processing CreateTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 (DEFAULT_TABLE table=test-table [id=43a7bcbe22334840b474ef6932d6959a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:40.745018 4288 tablet_service.cc:1467] Processing CreateTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 (DEFAULT_TABLE table=test-table [id=43a7bcbe22334840b474ef6932d6959a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:40.745445 4739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7788ad0e53b4a67b8ccb6afb59073f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:40.745857 4288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7788ad0e53b4a67b8ccb6afb59073f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:40.746237 4509 tablet_service.cc:1467] Processing CreateTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 (DEFAULT_TABLE table=test-table [id=43a7bcbe22334840b474ef6932d6959a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:40.747189 4509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7788ad0e53b4a67b8ccb6afb59073f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:40.751300 4889 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap starting.
I20260810 07:17:40.752709 4891 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Bootstrap starting.
I20260810 07:17:40.753692 4894 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Bootstrap starting.
I20260810 07:17:40.754782 4889 tablet_bootstrap.cc:654] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:40.754818 4891 tablet_bootstrap.cc:654] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:40.755726 4889 log.cc:824] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:40.755837 4894 tablet_bootstrap.cc:654] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:40.755996 4891 log.cc:824] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:40.756548 4894 log.cc:824] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:40.757804 4889 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: No bootstrap required, opened a new log
I20260810 07:17:40.758002 4889 ts_tablet_manager.cc:1397] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 07:17:40.758419 4894 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: No bootstrap required, opened a new log
I20260810 07:17:40.758598 4894 ts_tablet_manager.cc:1397] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:17:40.758634 4891 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: No bootstrap required, opened a new log
I20260810 07:17:40.758798 4891 ts_tablet_manager.cc:1397] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:17:40.761411 4894 raft_consensus.cc:348] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.761619 4894 raft_consensus.cc:374] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:40.761672 4894 raft_consensus.cc:729] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27aace958e024aec95a40bc13b35fc4f, State: Initialized, Role: FOLLOWER
I20260810 07:17:40.761788 4889 raft_consensus.cc:348] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.762050 4889 raft_consensus.cc:374] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:40.762104 4889 raft_consensus.cc:729] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6769c5a05d414bbc97f1c7c468e25f20, State: Initialized, Role: FOLLOWER
I20260810 07:17:40.762087 4894 consensus_queue.cc:260] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.762511 4889 consensus_queue.cc:260] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.762722 4624 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
I20260810 07:17:40.762995 4894 ts_tablet_manager.cc:1428] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:17:40.763473 4889 ts_tablet_manager.cc:1428] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:17:40.763545 4393 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
I20260810 07:17:40.763370 4891 raft_consensus.cc:348] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.763731 4891 raft_consensus.cc:374] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:40.763799 4891 raft_consensus.cc:729] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696bd1c5301e4638a086fac39cf822d0, State: Initialized, Role: FOLLOWER
I20260810 07:17:40.764520 4891 consensus_queue.cc:260] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.765486 4859 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
I20260810 07:17:40.765587 4891 ts_tablet_manager.cc:1428] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260810 07:17:40.795531 4394 tablet.cc:2369] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:40.939852 4905 raft_consensus.cc:482] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:40.940074 4905 raft_consensus.cc:504] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.941220 4905 leader_election.cc:288] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393), 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:40.944633 4888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.941533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:58799 (local address 127.12.117.129:38139)
0810 07:17:40.941712 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.941717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.941731 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:17:40.941784 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.941786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.941837 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.941915 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:40.942211 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.942708 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.943748 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.944128 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.944237 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.944290 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.944454 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.944519 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 07:17:40.944633 4861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.941416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.12.117.131:58799)
0810 07:17:40.941548 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.941564 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.941662 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.941944 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.941948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.941966 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:40.942087 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.942092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.942820 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.942823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.943615 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.943625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.944238 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.944242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.944346 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.944364 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.944429 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.944499 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"spinlock_wait_cycles":4992}
I20260810 07:17:40.944820 4886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.941552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:34037 (local address 127.12.117.130:44393)
0810 07:17:40.941697 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.941702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.941982 (+ 280us) server_negotiation.cc:407] Connection header received
0810 07:17:40.942095 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.942099 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.942144 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.942206 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:40.942457 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.942934 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.943957 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.944342 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.944444 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.944493 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.944653 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.944718 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:17:40.944835 4906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.941416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44393 (local address 127.12.117.131:34037)
0810 07:17:40.941886 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:40.941898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:40.941995 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:40.942228 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:40.942231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:40.942245 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:40.942353 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.942358 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.943039 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.943042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.943840 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:40.943849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.944444 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.944448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:40.944550 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.944568 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 07:17:40.944631 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:17:40.944695 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":109,"thread_start_us":146,"threads_started":1}
I20260810 07:17:40.945307 4320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7788ad0e53b4a67b8ccb6afb59073f1" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6769c5a05d414bbc97f1c7c468e25f20" is_pre_election: true
I20260810 07:17:40.945410 4542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7788ad0e53b4a67b8ccb6afb59073f1" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aace958e024aec95a40bc13b35fc4f" is_pre_election: true
I20260810 07:17:40.945577 4320 raft_consensus.cc:2454] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 0.
I20260810 07:17:40.945683 4542 raft_consensus.cc:2454] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 0.
I20260810 07:17:40.946002 4659 leader_election.cc:302] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20, 696bd1c5301e4638a086fac39cf822d0; no voters:
I20260810 07:17:40.946249 4905 raft_consensus.cc:2781] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:40.946326 4905 raft_consensus.cc:482] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:40.946357 4905 raft_consensus.cc:3037] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:40.947429 4905 raft_consensus.cc:504] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
W20260810 07:17:40.947572 4860 tablet.cc:2369] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:40.947785 4905 leader_election.cc:288] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 election: Requested vote from peers 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393), 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:40.948127 4542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7788ad0e53b4a67b8ccb6afb59073f1" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aace958e024aec95a40bc13b35fc4f"
I20260810 07:17:40.948237 4542 raft_consensus.cc:3037] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:40.949167 4542 raft_consensus.cc:2454] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 1.
I20260810 07:17:40.949397 4320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7788ad0e53b4a67b8ccb6afb59073f1" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6769c5a05d414bbc97f1c7c468e25f20"
I20260810 07:17:40.949445 4660 leader_election.cc:302] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0; no voters:
I20260810 07:17:40.949523 4320 raft_consensus.cc:3037] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:40.949591 4905 raft_consensus.cc:2781] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:40.949769 4905 raft_consensus.cc:686] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 1 LEADER]: Becoming Leader. State: Replica: 696bd1c5301e4638a086fac39cf822d0, State: Running, Role: LEADER
I20260810 07:17:40.950086 4905 consensus_queue.cc:237] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:40.950593 4320 raft_consensus.cc:2454] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 1.
I20260810 07:17:40.953006 4071 catalog_manager.cc:5495] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "696bd1c5301e4638a086fac39cf822d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } health_report { overall_health: UNKNOWN } } }
I20260810 07:17:40.979385 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:40.976388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47530 (local address 127.12.117.131:39919)
0810 07:17:40.976497 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:40.976501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:40.976513 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:17:40.976580 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:40.976582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:40.976631 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:40.976708 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:40.976950 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.977420 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.978199 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:40.978392 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:40.978466 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:40.978691 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:40.979070 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:40.979156 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:40.979200 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 07:17:40.979243 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:17:40.984541 4542 raft_consensus.cc:1261] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Refusing update from remote peer 696bd1c5301e4638a086fac39cf822d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:40.984541 4320 raft_consensus.cc:1261] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Refusing update from remote peer 696bd1c5301e4638a086fac39cf822d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:40.985168 4907 consensus_queue.cc:1035] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:40.985330 4905 consensus_queue.cc:1035] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:40.991983 4911 mvcc.cc:205] Tried to move back new op lower bound from 7316874284986290176 to 7316874284853858304. Current Snapshot: MvccSnapshot[applied={T|T < 7316874284986290176}]
W20260810 07:17:40.994581 4626 tablet.cc:2369] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:40.994725 4912 mvcc.cc:205] Tried to move back new op lower bound from 7316874284986290176 to 7316874284853858304. Current Snapshot: MvccSnapshot[applied={T|T < 7316874284986290176}]
I20260810 07:17:40.999455 4071 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:40.999614 4071 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:41.001850 4071 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Sending DeleteTablet for 3 replicas of tablet e7788ad0e53b4a67b8ccb6afb59073f1
I20260810 07:17:41.002519 4739 tablet_service.cc:1514] Processing DeleteTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:40 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:47522
I20260810 07:17:41.002556 4509 tablet_service.cc:1514] Processing DeleteTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:40 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:43724
I20260810 07:17:41.002552 4288 tablet_service.cc:1514] Processing DeleteTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:40 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55576
I20260810 07:17:41.002693 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 4172
I20260810 07:17:41.002951 4927 tablet_replica.cc:330] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: stopping tablet replica
I20260810 07:17:41.003326 4927 raft_consensus.cc:2229] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:41.003731 4927 raft_consensus.cc:2258] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:41.010216 4926 tablet_replica.cc:330] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: stopping tablet replica
I20260810 07:17:41.011296 4927 ts_tablet_manager.cc:1905] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:41.014318 4927 ts_tablet_manager.cc:1918] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:41.014678 4926 raft_consensus.cc:2229] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:41.015123 4926 raft_consensus.cc:2258] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:41.017714 4926 ts_tablet_manager.cc:1905] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:41.019376 4927 log.cc:1196] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/e7788ad0e53b4a67b8ccb6afb59073f1
I20260810 07:17:41.019767 4927 ts_tablet_manager.cc:1939] T e7788ad0e53b4a67b8ccb6afb59073f1 P 27aace958e024aec95a40bc13b35fc4f: Deleting consensus metadata
I20260810 07:17:41.020198 4926 ts_tablet_manager.cc:1918] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:41.020368 4926 log.cc:1196] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/e7788ad0e53b4a67b8ccb6afb59073f1
I20260810 07:17:41.020722 4926 ts_tablet_manager.cc:1939] T e7788ad0e53b4a67b8ccb6afb59073f1 P 696bd1c5301e4638a086fac39cf822d0: Deleting consensus metadata
I20260810 07:17:41.020985 4047 catalog_manager.cc:4841] TS 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): tablet e7788ad0e53b4a67b8ccb6afb59073f1 (table test-table [id=43a7bcbe22334840b474ef6932d6959a]) successfully deleted
I20260810 07:17:41.021903 4053 catalog_manager.cc:4841] TS 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): tablet e7788ad0e53b4a67b8ccb6afb59073f1 (table test-table [id=43a7bcbe22334840b474ef6932d6959a]) successfully deleted
I20260810 07:17:41.023373 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 4398
W20260810 07:17:41.023357 4046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:38139 (error 108)
I20260810 07:17:41.031294 4885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.030948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.0.0.1:55578)
0810 07:17:41.031084 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.031174 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 07:17:41.031522 4046 catalog_manager.cc:4584] TS 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e7788ad0e53b4a67b8ccb6afb59073f1: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38139: connect: Connection refused (error 111)
I20260810 07:17:41.036440 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 4632
I20260810 07:17:41.049235 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 4018
I20260810 07:17:41.060600 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--webserver_interface=127.12.117.190
--webserver_port=39033
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:41.167541 4929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.167769 4929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.167824 4929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.171694 4929 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:41.171739 4929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.171765 4929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.171787 4929 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:41.171816 4929 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:41.176802 4929 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=39033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:41.177732 4929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.178630 4929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:17:41.191215 4929 server_base.cc:1034] running on GCE node
W20260810 07:17:41.191177 4939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.191195 4940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.191177 4943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.191999 4929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.193037 4929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.194197 4929 hybrid_clock.cc:648] HybridClock initialized: now 1786346261194178 us; error 29 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.196856 4929 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:41.197680 4929 webserver.cc:460] Webserver started at http://127.12.117.190:39033/ using document root <none> and password file <none>
I20260810 07:17:41.198205 4929 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.198283 4929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.201881 4929 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.203863 4955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.204986 4929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.205107 4929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "11abe7d525dc431fa5e70f480de55c90"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "de2f3691bdd3924c699d0069bfd01825"
server_key_iv: "db87e5904ec75e6055a8a62f38cf30ce"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.205631 4929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.243856 4929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.246732 4929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.246899 4929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.253247 4929 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39829
I20260810 07:17:41.253502 5038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39829 every 8 connection(s)
I20260810 07:17:41.254243 4929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:41.256374 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 4929
I20260810 07:17:41.256841 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:38139
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=45723
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:41.258908 5041 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:41.260499 5041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap starting.
I20260810 07:17:41.276403 5041 log.cc:824] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.284545 5041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:41.284843 5041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap complete.
I20260810 07:17:41.288604 5041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:41.288969 5041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.289435 5041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:41.289546 5041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:41.289611 5041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:41.289698 5041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:17:41.291523 5041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:41.291818 5041 leader_election.cc:302] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11abe7d525dc431fa5e70f480de55c90; no voters:
I20260810 07:17:41.292092 5041 leader_election.cc:288] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:17:41.292199 5047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:17:41.292433 5047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 LEADER]: Becoming Leader. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Running, Role: LEADER
I20260810 07:17:41.292770 5047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:41.293171 5041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:41.293706 5047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11abe7d525dc431fa5e70f480de55c90. Latest consensus state: current_term: 2 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:41.293893 5047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:41.294059 5051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:41.294229 5051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:41.294442 5061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:41.298966 5061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:41.299867 5061 catalog_manager.cc:1259] Loaded cluster ID: 1997d02d1c004252a74c18c8fdfc01f3
I20260810 07:17:41.299933 5061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:41.301379 5061 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 07:17:41.305894 5061 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 07:17:41.310606 5061 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:41.311298 5061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Loaded TSK: 0
I20260810 07:17:41.312475 5061 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:17:41.369760 5044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.369977 5044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.370036 5044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.373872 5044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.373922 5044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.374003 5044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:41.378891 5044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:38139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=45723
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:41.379776 5044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.380640 5044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.390364 5087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.390358 5088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.390525 5044 server_base.cc:1034] running on GCE node
W20260810 07:17:41.390372 5090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.391090 5044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.391815 5044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.393015 5044 hybrid_clock.cc:648] HybridClock initialized: now 1786346261392996 us; error 51 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.396382 5044 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:41.397403 5044 webserver.cc:460] Webserver started at http://127.12.117.129:45723/ using document root <none> and password file <none>
I20260810 07:17:41.398093 5044 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.398203 5044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.402956 5044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:41.405436 5105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.406778 5044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:17:41.406895 5044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "6769c5a05d414bbc97f1c7c468e25f20"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "83ff3a6b5a6e42bc4ff28b2c051e09b8"
server_key_iv: "0543e7b60415bb50a39daec4f544f3ff"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.407568 5044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.428643 5044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.431831 5044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.431993 5044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.432564 5044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.433673 5115 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:17:41.437567 5044 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:17:41.437636 5044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260810 07:17:41.437687 5044 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:17:41.439646 5044 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:17:41.439703 5044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.439961 5115 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap starting.
I20260810 07:17:41.449131 5044 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38139
I20260810 07:17:41.449235 5287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38139 every 8 connection(s)
I20260810 07:17:41.449993 5044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:41.453594 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 5044
I20260810 07:17:41.454085 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:44393
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=38465
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:41.458506 5115 log.cc:824] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:41.466253 5115 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:41.466626 5115 tablet_bootstrap.cc:492] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap complete.
I20260810 07:17:41.467792 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.451831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:56275 (local address 127.12.117.190:39829)
0810 07:17:41.452160 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.452176 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.452615 (+ 439us) server_negotiation.cc:407] Connection header received
0810 07:17:41.453642 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.453653 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.453967 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.454268 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.455426 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.456711 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.458335 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.458880 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.459059 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.462105 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.462112 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.462128 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.462162 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.464221 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.464866 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.464871 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.464877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.465025 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.465423 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.465426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.465428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.465868 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.465997 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.466675 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.466994 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260810 07:17:41.467859 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.451924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:56275)
0810 07:17:41.452462 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.452501 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.453243 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.454339 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.454347 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.454799 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.455231 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.455244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.456890 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.456894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.458207 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.458220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.459028 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.459032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.459252 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.459915 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.459941 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.461863 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.464354 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.464361 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.464375 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.464740 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.465128 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.465131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.465133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.465319 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.465993 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.466000 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.466309 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.466526 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.466889 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":2,"thread_start_us":188,"threads_started":1}
I20260810 07:17:41.467953 5115 ts_tablet_manager.cc:1397] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent bootstrapping tablet: real 0.028s user 0.023s sys 0.004s
I20260810 07:17:41.470489 5288 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:41.470727 5288 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.470386 5115 raft_consensus.cc:348] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.470855 5115 raft_consensus.cc:729] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6769c5a05d414bbc97f1c7c468e25f20, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.471235 5288 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
I20260810 07:17:41.471253 5115 consensus_queue.cc:260] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.472059 5115 ts_tablet_manager.cc:1428] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260810 07:17:41.473598 4981 ts_manager.cc:194] Registered new tserver with Master: 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:41.476627 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:56275
I20260810 07:17:41.478775 5288 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:17:41.486363 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.476274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.12.117.129:38139)
0810 07:17:41.476895 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.476905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.477123 (+ 218us) server_negotiation.cc:407] Connection header received
0810 07:17:41.477298 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.477303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.477492 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.477623 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.478559 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.479671 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.480711 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.481183 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.481293 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.483363 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.483371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.483381 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.483401 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.484684 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.485192 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.485196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.485197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.485264 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.485516 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.485518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.485519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.485707 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.485797 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.486130 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.486250 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":784,"server-negotiator.queue_time_us":531,"thread_start_us":88,"threads_started":1}
I20260810 07:17:41.486418 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.476188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.0.0.1:55594)
0810 07:17:41.476990 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.477030 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.477166 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.477885 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.477891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.478291 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.478411 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.478419 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.479808 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.479812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.480603 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.480612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.481282 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.481285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.481357 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.481810 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.481821 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.483192 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.484792 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.484801 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.484806 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.485098 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.485341 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.485344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.485345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.485440 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.485804 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.485810 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.485933 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.486074 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.486258 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":88,"threads_started":1}
I20260810 07:17:41.487355 5169 tablet_service.cc:1514] Processing DeleteTablet for tablet e7788ad0e53b4a67b8ccb6afb59073f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:40 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55594
I20260810 07:17:41.488233 5303 tablet_replica.cc:330] stopping tablet replica
I20260810 07:17:41.488476 5303 raft_consensus.cc:2229] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:41.488690 5303 raft_consensus.cc:2258] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:41.489495 5303 ts_tablet_manager.cc:1905] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:41.493464 5303 ts_tablet_manager.cc:1918] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:41.493562 5303 log.cc:1196] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/e7788ad0e53b4a67b8ccb6afb59073f1
I20260810 07:17:41.493891 5303 ts_tablet_manager.cc:1939] T e7788ad0e53b4a67b8ccb6afb59073f1 P 6769c5a05d414bbc97f1c7c468e25f20: Deleting consensus metadata
I20260810 07:17:41.495744 4958 catalog_manager.cc:4841] TS 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139): tablet e7788ad0e53b4a67b8ccb6afb59073f1 (table test-table [id=43a7bcbe22334840b474ef6932d6959a]) successfully deleted
W20260810 07:17:41.572026 5296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.572265 5296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.572324 5296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.576196 5296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.576244 5296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.576319 5296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:41.581272 5296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:44393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=38465
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:41.582181 5296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.583050 5296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.591905 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.591894 5311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.592065 5296 server_base.cc:1034] running on GCE node
W20260810 07:17:41.591894 5312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.592533 5296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.593101 5296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.594251 5296 hybrid_clock.cc:648] HybridClock initialized: now 1786346261594235 us; error 27 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.596720 5296 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:41.597445 5296 webserver.cc:460] Webserver started at http://127.12.117.130:38465/ using document root <none> and password file <none>
I20260810 07:17:41.597919 5296 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.597996 5296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.601462 5296 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:17:41.603320 5326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.604367 5296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.604466 5296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "27aace958e024aec95a40bc13b35fc4f"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "4d5f0b5cf8d0e53d6bd6f615c7ebf6d1"
server_key_iv: "c45e74e16ac70eb34bd3c0c73a54fba5"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.604928 5296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.652673 5296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.655087 5296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.655249 5296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.655797 5296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.656529 5296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:41.656584 5296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.656633 5296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:41.656657 5296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.665591 5296 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44393
I20260810 07:17:41.665620 5519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44393 every 8 connection(s)
I20260810 07:17:41.666464 5296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:41.673158 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 5296
I20260810 07:17:41.673599 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:39919
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=33369
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:41.679698 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.668137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:54547 (local address 127.12.117.190:39829)
0810 07:17:41.668273 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.668277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.669026 (+ 749us) server_negotiation.cc:407] Connection header received
0810 07:17:41.670342 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.670345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.670402 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.670477 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.671615 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.672169 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.673460 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.673741 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.673852 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.676184 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.676188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.676191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.676221 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.677623 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.678130 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.678134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.678136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.678225 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.678588 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.678592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.678594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.678830 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.678940 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.679386 (+ 446us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.679523 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:17:41.680315 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.668470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:54547)
0810 07:17:41.668890 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.668930 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.670139 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.670662 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.670669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.670992 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.671478 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.671493 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.672292 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.672295 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.673341 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.673353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.673888 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.673893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.674063 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.674559 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.674574 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.676011 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.677739 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.677743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.677754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.677995 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.678350 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.678353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.678355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.678477 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.678926 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.678930 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.679143 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.679318 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.679554 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260810 07:17:41.681383 5520 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:41.681591 5520 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.682032 5520 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
I20260810 07:17:41.682964 4981 ts_manager.cc:194] Registered new tserver with Master: 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
I20260810 07:17:41.683545 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:54547
W20260810 07:17:41.780920 5524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:41.781140 5524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:41.781198 5524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:41.785089 5524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:41.785138 5524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:41.785211 5524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:17:41.790284 5524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:39919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=33369
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:41.791133 5524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:41.792012 5524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:41.801142 5530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.801136 5535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:41.801132 5533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:41.801416 5524 server_base.cc:1034] running on GCE node
I20260810 07:17:41.801728 5524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:41.802287 5524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:41.803432 5524 hybrid_clock.cc:648] HybridClock initialized: now 1786346261803416 us; error 26 us; skew 500 ppm
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:41.805821 5524 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:17:41.806595 5524 webserver.cc:460] Webserver started at http://127.12.117.131:33369/ using document root <none> and password file <none>
I20260810 07:17:41.807075 5524 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:41.807148 5524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:41.810917 5524 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:17:41.812906 5546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.814057 5524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:17:41.814147 5524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "696bd1c5301e4638a086fac39cf822d0"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "1be580634f709c709edbd740b5bfb758"
server_key_iv: "77107937d5520bd9875c7da5645e51ca"
server_key_version: "encryptionkey@0"
I20260810 07:17:41.814610 5524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:41.838407 5524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:41.840899 5524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:41.841038 5524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:41.841511 5524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:41.842264 5524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:41.842308 5524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.842344 5524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:41.842363 5524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:41.850673 5524 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:39919
I20260810 07:17:41.850699 5736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:39919 every 8 connection(s)
I20260810 07:17:41.851586 5524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
Aug 10 07:17:41 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346261, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:41.860400 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 5524
I20260810 07:17:41.864491 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.853192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35733 (local address 127.12.117.190:39829)
0810 07:17:41.853307 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.853311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.854076 (+ 765us) server_negotiation.cc:407] Connection header received
0810 07:17:41.855404 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.855408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.855462 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.855536 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.856620 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.857131 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.858212 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.858430 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.858517 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.860865 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.860869 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.860872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.860903 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.862490 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.863051 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.863054 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.863055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.863129 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.863485 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.863489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.863491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.863720 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.863822 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.864260 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.864369 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:41.865265 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.853528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:35733)
0810 07:17:41.853947 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.853985 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.855193 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.855717 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.855724 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.856042 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:41.856475 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.856488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.857245 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.857249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.858092 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.858103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.858561 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.858566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.858730 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.859168 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:41.859183 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:41.860685 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:41.862595 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.862601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.862625 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.862961 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.863206 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:41.863209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:41.863211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:41.863401 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:41.863816 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:41.863820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:41.864029 (+ 209us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.864173 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.864452 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260810 07:17:41.866433 5737 heartbeater.cc:347] Connected to a master server at 127.12.117.190:39829
I20260810 07:17:41.866688 5737 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:41.867233 5737 heartbeater.cc:511] Master 127.12.117.190:39829 requested a full tablet report, sending...
I20260810 07:17:41.868400 4981 ts_manager.cc:194] Registered new tserver with Master: 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919)
I20260810 07:17:41.869074 4981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:35733
I20260810 07:17:41.869246 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.861048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.12.117.190:39829)
0810 07:17:41.861285 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.861290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.861455 (+ 165us) server_negotiation.cc:407] Connection header received
0810 07:17:41.861584 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.861586 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.861628 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.861710 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:41.862591 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.863320 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.864309 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.864649 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.864756 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.865952 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:41.865955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:41.865956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:41.865978 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:41.867661 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.868123 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.868126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.868128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.868216 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:41.868490 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:41.868492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:41.868494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:41.868670 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:41.868809 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.869003 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.869096 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
I20260810 07:17:41.871120 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:17:41.875046 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.871701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.12.117.190:39829)
0810 07:17:41.871813 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.871817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.871982 (+ 165us) server_negotiation.cc:407] Connection header received
0810 07:17:41.872146 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.872150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.872231 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.872311 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:41.872542 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.873009 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.873802 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.874029 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.874122 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.874405 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:41.874713 (+ 308us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:41.874811 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.874887 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.874939 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:17:41.875602 4981 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:17:41.878662 4981 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:17:41.885797 5169 tablet_service.cc:1467] Processing CreateTablet for tablet f4284889905748dca6393b8d2eef3971 (DEFAULT_TABLE table=test-table [id=360551b6c02a4a668bf6629f8423dd58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.886343 5169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4284889905748dca6393b8d2eef3971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.889112 5115 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap starting.
I20260810 07:17:41.889142 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.884774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:39919 (local address 127.0.0.1:47544)
0810 07:17:41.884922 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.884935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.885006 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.885729 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.885732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.885751 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:41.885835 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.885841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.887049 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.887052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.887962 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.887972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.888724 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.888728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.888829 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.888870 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.888944 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.889023 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 07:17:41.889405 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.884976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.12.117.131:39919)
0810 07:17:41.885283 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.885294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.885317 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:17:41.885383 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.885388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.885573 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.885734 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:41.885932 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.886908 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.888083 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.888604 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.888723 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.888845 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.888982 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.889067 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":120,"threads_started":1}
I20260810 07:17:41.889961 5115 tablet_bootstrap.cc:654] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.891394 5115 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: No bootstrap required, opened a new log
I20260810 07:17:41.891484 5115 ts_tablet_manager.cc:1397] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:41.891635 5755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.885120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44393 (local address 127.0.0.1:43732)
0810 07:17:41.885332 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:41.885345 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:41.885426 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:41.886415 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:41.886418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:41.886437 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:41.886518 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.886524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.888141 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.888145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.889911 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:41.889923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.891217 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.891221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:41.891342 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.891365 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:17:41.891448 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 07:17:41.891517 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260810 07:17:41.891875 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:41.885509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43732 (local address 127.12.117.130:44393)
0810 07:17:41.885962 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:41.885977 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:41.885998 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:17:41.886070 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:41.886075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:41.886264 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:41.886408 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:41.886655 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.887987 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.890172 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:41.890901 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:41.891014 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:41.891135 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:41.891509 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 07:17:41.891605 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":131,"threads_started":1}
I20260810 07:17:41.891899 5115 raft_consensus.cc:348] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.892024 5115 raft_consensus.cc:374] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.892067 5115 raft_consensus.cc:729] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6769c5a05d414bbc97f1c7c468e25f20, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.892186 5115 consensus_queue.cc:260] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.892446 5115 ts_tablet_manager.cc:1428] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:17:41.892094 5625 tablet_service.cc:1467] Processing CreateTablet for tablet f4284889905748dca6393b8d2eef3971 (DEFAULT_TABLE table=test-table [id=360551b6c02a4a668bf6629f8423dd58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.893103 5625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4284889905748dca6393b8d2eef3971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.894642 5407 tablet_service.cc:1467] Processing CreateTablet for tablet f4284889905748dca6393b8d2eef3971 (DEFAULT_TABLE table=test-table [id=360551b6c02a4a668bf6629f8423dd58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:17:41.895596 5407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4284889905748dca6393b8d2eef3971. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:41.900270 5758 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Bootstrap starting.
I20260810 07:17:41.902294 5759 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Bootstrap starting.
I20260810 07:17:41.902962 5758 tablet_bootstrap.cc:654] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.903749 5758 log.cc:824] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.904429 5759 tablet_bootstrap.cc:654] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:41.905077 5759 log.cc:824] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:41.905467 5758 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: No bootstrap required, opened a new log
I20260810 07:17:41.905664 5758 ts_tablet_manager.cc:1397] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:17:41.906653 5759 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: No bootstrap required, opened a new log
I20260810 07:17:41.906821 5759 ts_tablet_manager.cc:1397] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:17:41.908812 5758 raft_consensus.cc:348] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.909025 5758 raft_consensus.cc:374] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.909080 5758 raft_consensus.cc:729] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696bd1c5301e4638a086fac39cf822d0, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.909480 5758 consensus_queue.cc:260] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.910118 5737 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
I20260810 07:17:41.910285 5758 ts_tablet_manager.cc:1428] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:17:41.910194 5759 raft_consensus.cc:348] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.910431 5759 raft_consensus.cc:374] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:41.910486 5759 raft_consensus.cc:729] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27aace958e024aec95a40bc13b35fc4f, State: Initialized, Role: FOLLOWER
I20260810 07:17:41.910882 5759 consensus_queue.cc:260] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:41.911561 5520 heartbeater.cc:503] Master 127.12.117.190:39829 was elected leader, sending a full tablet report...
I20260810 07:17:41.911844 5759 ts_tablet_manager.cc:1428] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 07:17:41.917232 5521 tablet.cc:2369] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:17:41.951068 5291 tablet.cc:2369] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:42.075183 5765 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:42.075397 5765 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:42.076511 5765 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393), 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:42.080282 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.076830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:33461 (local address 127.12.117.129:38139)
0810 07:17:42.077141 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.077145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.077160 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:17:42.077216 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.077219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.077285 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.077389 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:42.077695 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.078215 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.079316 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.079747 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.079860 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.080022 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.080093 (+ 71us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.080171 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":112,"threads_started":1}
I20260810 07:17:42.080281 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.076741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.12.117.131:33461)
0810 07:17:42.076849 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.076866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:42.076954 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:42.077416 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:42.077420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:42.077467 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:42.077572 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.077578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.078328 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.078331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.079163 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.079174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.079865 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.079869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.079972 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.079995 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:17:42.080069 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 07:17:42.080140 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"spinlock_wait_cycles":38528}
I20260810 07:17:42.080461 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.076847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:52033 (local address 127.12.117.130:44393)
0810 07:17:42.076983 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.076988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.077434 (+ 446us) server_negotiation.cc:407] Connection header received
0810 07:17:42.077583 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.077586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.077628 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.077688 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:17:42.077933 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.078404 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.079544 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.079946 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.080052 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.080103 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.080289 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.080354 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:17:42.080477 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.076749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44393 (local address 127.12.117.131:52033)
0810 07:17:42.077344 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.077356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:42.077478 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:42.077709 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:42.077712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:42.077727 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:17:42.077833 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.077839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.078503 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.078506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.079426 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:42.079436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.080059 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.080062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:42.080175 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.080197 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:17:42.080260 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:17:42.080329 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":273,"spinlock_wait_cycles":5376,"thread_start_us":135,"threads_started":1}
I20260810 07:17:42.080962 5204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4284889905748dca6393b8d2eef3971" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6769c5a05d414bbc97f1c7c468e25f20" is_pre_election: true
I20260810 07:17:42.081012 5441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4284889905748dca6393b8d2eef3971" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aace958e024aec95a40bc13b35fc4f" is_pre_election: true
I20260810 07:17:42.081251 5204 raft_consensus.cc:2454] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 0.
I20260810 07:17:42.081285 5441 raft_consensus.cc:2454] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 0.
I20260810 07:17:42.081687 5549 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20, 696bd1c5301e4638a086fac39cf822d0; no voters:
I20260810 07:17:42.081954 5765 raft_consensus.cc:2781] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:17:42.082041 5765 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:17:42.082094 5765 raft_consensus.cc:3037] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.083086 5765 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:42.083428 5765 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 election: Requested vote from peers 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393), 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139)
I20260810 07:17:42.083782 5441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4284889905748dca6393b8d2eef3971" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27aace958e024aec95a40bc13b35fc4f"
I20260810 07:17:42.083773 5204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4284889905748dca6393b8d2eef3971" candidate_uuid: "696bd1c5301e4638a086fac39cf822d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6769c5a05d414bbc97f1c7c468e25f20"
I20260810 07:17:42.083892 5204 raft_consensus.cc:3037] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.083892 5441 raft_consensus.cc:3037] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:42.085094 5441 raft_consensus.cc:2454] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 1.
I20260810 07:17:42.085094 5204 raft_consensus.cc:2454] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696bd1c5301e4638a086fac39cf822d0 in term 1.
I20260810 07:17:42.085358 5549 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20, 696bd1c5301e4638a086fac39cf822d0; no voters:
I20260810 07:17:42.085520 5765 raft_consensus.cc:2781] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:42.085711 5765 raft_consensus.cc:686] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 1 LEADER]: Becoming Leader. State: Replica: 696bd1c5301e4638a086fac39cf822d0, State: Running, Role: LEADER
I20260810 07:17:42.086035 5765 consensus_queue.cc:237] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:42.089015 4980 catalog_manager.cc:5495] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "696bd1c5301e4638a086fac39cf822d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } health_report { overall_health: UNKNOWN } } }
W20260810 07:17:42.102404 5738 tablet.cc:2369] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:17:42.121986 5751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.118365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.12.117.131:39919)
0810 07:17:42.118481 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:42.118486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:42.118562 (+ 76us) server_negotiation.cc:407] Connection header received
0810 07:17:42.118727 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:42.118730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:42.118784 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:42.118870 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:17:42.119115 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.119625 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.120435 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:42.120729 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:42.120845 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:42.121164 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:17:42.121560 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:17:42.121670 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:42.121751 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 07:17:42.121807 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:17:42.127389 5204 raft_consensus.cc:1261] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Refusing update from remote peer 696bd1c5301e4638a086fac39cf822d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:42.127393 5441 raft_consensus.cc:1261] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Refusing update from remote peer 696bd1c5301e4638a086fac39cf822d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:17:42.127980 5765 consensus_queue.cc:1035] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:42.128173 5772 consensus_queue.cc:1035] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:17:42.133476 5777 mvcc.cc:205] Tried to move back new op lower bound from 7316874289667334144 to 7316874289507053568. Current Snapshot: MvccSnapshot[applied={T|T < 7316874289667334144}]
I20260810 07:17:42.134261 5778 mvcc.cc:205] Tried to move back new op lower bound from 7316874289667334144 to 7316874289507053568. Current Snapshot: MvccSnapshot[applied={T|T < 7316874289667334144}]
I20260810 07:17:42.141909 4980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:42.142042 4980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:17:42.144169 4980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Sending DeleteTablet for 3 replicas of tablet f4284889905748dca6393b8d2eef3971
I20260810 07:17:42.144770 5625 tablet_service.cc:1514] Processing DeleteTablet for tablet f4284889905748dca6393b8d2eef3971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:47544
I20260810 07:17:42.144834 5169 tablet_service.cc:1514] Processing DeleteTablet for tablet f4284889905748dca6393b8d2eef3971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55594
I20260810 07:17:42.144886 5407 tablet_service.cc:1514] Processing DeleteTablet for tablet f4284889905748dca6393b8d2eef3971 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:17:42 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:43732
I20260810 07:17:42.145364 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 5044
I20260810 07:17:42.160204 5802 tablet_replica.cc:330] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: stopping tablet replica
I20260810 07:17:42.160518 5802 raft_consensus.cc:2229] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:17:42.160836 5802 raft_consensus.cc:2258] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:42.162309 5802 ts_tablet_manager.cc:1905] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:17:42.164567 5802 ts_tablet_manager.cc:1918] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:42.164673 5802 log.cc:1196] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/f4284889905748dca6393b8d2eef3971
I20260810 07:17:42.164951 5802 ts_tablet_manager.cc:1939] T f4284889905748dca6393b8d2eef3971 P 27aace958e024aec95a40bc13b35fc4f: Deleting consensus metadata
I20260810 07:17:42.166015 5800 tablet_replica.cc:330] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: stopping tablet replica
I20260810 07:17:42.166093 4959 catalog_manager.cc:4841] TS 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): tablet f4284889905748dca6393b8d2eef3971 (table test-table [id=360551b6c02a4a668bf6629f8423dd58]) successfully deleted
I20260810 07:17:42.166358 5800 raft_consensus.cc:2229] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:17:42.166687 5800 raft_consensus.cc:2258] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:17:42.167971 5800 ts_tablet_manager.cc:1905] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 07:17:42.168128 4958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:38139 (error 108)
W20260810 07:17:42.169404 5755 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:17:42.169077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:38139 (local address 127.0.0.1:55596)
0810 07:17:42.169225 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.169302 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38139: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":60}
W20260810 07:17:42.169545 4958 catalog_manager.cc:4584] TS 6769c5a05d414bbc97f1c7c468e25f20 (127.12.117.129:38139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4284889905748dca6393b8d2eef3971: Network error: Client connection negotiation failed: client connection to 127.12.117.129:38139: connect: Connection reset by peer (error 104)
I20260810 07:17:42.170360 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 5296
I20260810 07:17:42.170423 5800 ts_tablet_manager.cc:1918] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:17:42.170500 5800 log.cc:1196] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/f4284889905748dca6393b8d2eef3971
I20260810 07:17:42.170825 5800 ts_tablet_manager.cc:1939] T f4284889905748dca6393b8d2eef3971 P 696bd1c5301e4638a086fac39cf822d0: Deleting consensus metadata
I20260810 07:17:42.172741 4960 catalog_manager.cc:4841] TS 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): tablet f4284889905748dca6393b8d2eef3971 (table test-table [id=360551b6c02a4a668bf6629f8423dd58]) successfully deleted
I20260810 07:17:42.183947 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 5524
I20260810 07:17:42.207568 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 4929
I20260810 07:17:42.222117 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--webserver_interface=127.12.117.190
--webserver_port=39033
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:42.329885 5808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:42.330137 5808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:42.330193 5808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:42.334162 5808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:42.334215 5808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:42.334241 5808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:42.334261 5808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:42.334286 5808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:42.339435 5808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=39033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:42.340348 5808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:42.341264 5808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:42.349123 5823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.349130 5819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.349244 5808 server_base.cc:1034] running on GCE node
W20260810 07:17:42.349130 5817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.349912 5808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:42.350929 5808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:42.352078 5808 hybrid_clock.cc:648] HybridClock initialized: now 1786346262352065 us; error 25 us; skew 500 ppm
Aug 10 07:17:42 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:42.354624 5808 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:42.355515 5808 webserver.cc:460] Webserver started at http://127.12.117.190:39033/ using document root <none> and password file <none>
I20260810 07:17:42.356060 5808 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:42.356140 5808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:42.359925 5808 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 07:17:42.361901 5834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.363044 5808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:17:42.363161 5808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "11abe7d525dc431fa5e70f480de55c90"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "de2f3691bdd3924c699d0069bfd01825"
server_key_iv: "db87e5904ec75e6055a8a62f38cf30ce"
server_key_version: "encryptionkey@0"
I20260810 07:17:42.363716 5808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:42.385761 5808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:42.388387 5808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:42.388556 5808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:42.395013 5808 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39829
I20260810 07:17:42.395257 5931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39829 every 8 connection(s)
I20260810 07:17:42.396075 5808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:42.397773 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 5808
I20260810 07:17:42.398154 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:38139
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=45723
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:42.400390 5932 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:17:42.402192 5932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap starting.
I20260810 07:17:42.418555 5932 log.cc:824] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:42.429117 5932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:42.429420 5932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90: Bootstrap complete.
I20260810 07:17:42.433090 5932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:42.433456 5932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Initialized, Role: FOLLOWER
I20260810 07:17:42.433895 5932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:42.434003 5932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:42.434060 5932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:42.434154 5932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:17:42.435961 5932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:42.436266 5932 leader_election.cc:302] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11abe7d525dc431fa5e70f480de55c90; no voters:
I20260810 07:17:42.436528 5932 leader_election.cc:288] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:17:42.436617 5939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:17:42.436801 5939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [term 3 LEADER]: Becoming Leader. State: Replica: 11abe7d525dc431fa5e70f480de55c90, State: Running, Role: LEADER
I20260810 07:17:42.437142 5939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } }
I20260810 07:17:42.437693 5932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:42.438063 5939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11abe7d525dc431fa5e70f480de55c90. Latest consensus state: current_term: 3 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:42.438190 5939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:42.439344 5940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "11abe7d525dc431fa5e70f480de55c90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11abe7d525dc431fa5e70f480de55c90" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39829 } } }
I20260810 07:17:42.439512 5940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11abe7d525dc431fa5e70f480de55c90 [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:42.439659 5944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:42.445297 5944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:42.445917 5944 catalog_manager.cc:1259] Loaded cluster ID: 1997d02d1c004252a74c18c8fdfc01f3
I20260810 07:17:42.445979 5944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 07:17:42.446962 5944 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd9dc64b9ad google::LogMessage::Fail() at ??:0
@ 0x7fd9dc64e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd9dc64b4de google::LogMessage::Flush() at ??:0
@ 0x7fd9dc64ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd9e6f8afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd9e6f8bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd9e6f88579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd9e6fcc640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd9de89d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd9de89dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd9de89f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd9de88e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd9dfed26db start_thread at ??:0
@ 0x7fd9db14961f clone at ??:0
W20260810 07:17:42.532778 5936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:42.532994 5936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:42.533032 5936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:42.536901 5936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:42.536949 5936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:42.537005 5936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:42.541998 5936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:38139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=45723
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:42.542919 5936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:42.543895 5936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:42.553001 5996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.553043 6003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.552953 6000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.553421 5936 server_base.cc:1034] running on GCE node
I20260810 07:17:42.553771 5936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:42.554325 5936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:42.555472 5936 hybrid_clock.cc:648] HybridClock initialized: now 1786346262555442 us; error 48 us; skew 500 ppm
Aug 10 07:17:42 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:42.558219 5936 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:42.559239 5936 webserver.cc:460] Webserver started at http://127.12.117.129:45723/ using document root <none> and password file <none>
I20260810 07:17:42.559911 5936 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:42.559999 5936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:42.564792 5936 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:17:42.567242 6013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.568636 5936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:42.568758 5936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "6769c5a05d414bbc97f1c7c468e25f20"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "83ff3a6b5a6e42bc4ff28b2c051e09b8"
server_key_iv: "0543e7b60415bb50a39daec4f544f3ff"
server_key_version: "encryptionkey@0"
I20260810 07:17:42.569398 5936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:42.606745 5936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:42.610095 5936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:42.610291 5936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:42.610970 5936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:42.612324 6025 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:17:42.616365 5936 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:17:42.616458 5936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260810 07:17:42.616528 5936 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:17:42.618649 5936 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:17:42.618708 5936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 07:17:42.618960 6025 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap starting.
I20260810 07:17:42.629222 5936 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:38139
I20260810 07:17:42.630328 5936 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:42.631556 6208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:38139 every 8 connection(s)
I20260810 07:17:42.635632 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 5936
I20260810 07:17:42.636148 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:44393
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=38465
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:17:42.639547 6025 log.cc:824] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:42.648144 6025 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:17:42.649190 6025 tablet_bootstrap.cc:492] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Bootstrap complete.
I20260810 07:17:42.650287 6025 ts_tablet_manager.cc:1397] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent bootstrapping tablet: real 0.032s user 0.028s sys 0.000s
I20260810 07:17:42.652777 6025 raft_consensus.cc:348] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:42.653232 6025 raft_consensus.cc:729] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6769c5a05d414bbc97f1c7c468e25f20, State: Initialized, Role: FOLLOWER
I20260810 07:17:42.653668 6025 consensus_queue.cc:260] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:42.655123 6025 ts_tablet_manager.cc:1428] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260810 07:17:42.661702 6212 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:17:42.632464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:33769)
0810 07:17:42.632935 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.632985 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:42.633969 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:42.660821 (+ 26852us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":130,"threads_started":1}
W20260810 07:17:42.662159 6209 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:39829 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:39829: Client connection negotiation failed: client connection to 127.12.117.190:39829: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:17:42.662834 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.662562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:57847)
0810 07:17:42.662696 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.662771 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 07:17:42.663465 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.663233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:49087)
0810 07:17:42.663355 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.663404 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 07:17:42.663635 6209 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:17:42.752120 6213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:42.752326 6213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:42.752363 6213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:42.756170 6213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:42.756215 6213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:42.756268 6213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:42.761124 6213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:44393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=38465
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:42.761933 6213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:42.762760 6213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:42.771509 6231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.771564 6234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.771565 6230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.771659 6213 server_base.cc:1034] running on GCE node
I20260810 07:17:42.772106 6213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:42.772639 6213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:42.773788 6213 hybrid_clock.cc:648] HybridClock initialized: now 1786346262773776 us; error 25 us; skew 500 ppm
Aug 10 07:17:42 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:42.776309 6213 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:42.777040 6213 webserver.cc:460] Webserver started at http://127.12.117.130:38465/ using document root <none> and password file <none>
I20260810 07:17:42.777513 6213 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:42.777570 6213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:42.781042 6213 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:17:42.782871 6248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.783993 6213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:42.784106 6213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "27aace958e024aec95a40bc13b35fc4f"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "4d5f0b5cf8d0e53d6bd6f615c7ebf6d1"
server_key_iv: "c45e74e16ac70eb34bd3c0c73a54fba5"
server_key_version: "encryptionkey@0"
I20260810 07:17:42.784566 6213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:42.814435 6213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:42.817307 6213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:42.817467 6213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:42.818032 6213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:42.818801 6213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:42.818849 6213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.818890 6213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:42.818918 6213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.827088 6213 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:44393
I20260810 07:17:42.827137 6457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:44393 every 8 connection(s)
I20260810 07:17:42.828058 6213 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:42.831750 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.829987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:58173)
0810 07:17:42.830434 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.830542 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":120,"threads_started":1}
W20260810 07:17:42.832147 6458 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:39829 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:39829: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
I20260810 07:17:42.832671 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.832476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:39505)
0810 07:17:42.832568 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.832615 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:17:42.833173 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:42.833006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:46045)
0810 07:17:42.833082 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:42.833126 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:17:42.833304 6458 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:17:42.838157 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 6213
I20260810 07:17:42.838537 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:39919
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:39829
--webserver_port=33369
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:42.946571 6465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:42.946805 6465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:42.946862 6465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:42.950860 6465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:42.950914 6465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:42.950999 6465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:17:42.956060 6465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:39919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=33369
--tserver_master_addrs=127.12.117.190:39829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:42.956959 6465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:42.957875 6465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:42.967031 6474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.967139 6465 server_base.cc:1034] running on GCE node
W20260810 07:17:42.967024 6473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:42.967026 6476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:42.967716 6465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:42.968405 6465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:42.969554 6465 hybrid_clock.cc:648] HybridClock initialized: now 1786346262969533 us; error 35 us; skew 500 ppm
Aug 10 07:17:42 dist-test-slave-gjrz krb5kdc[3997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:42.971966 6465 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:17:42.972757 6465 webserver.cc:460] Webserver started at http://127.12.117.131:33369/ using document root <none> and password file <none>
I20260810 07:17:42.973246 6465 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:42.973302 6465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:42.977224 6465 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:17:42.979072 6495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.980274 6465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:42.980373 6465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "696bd1c5301e4638a086fac39cf822d0"
format_stamp: "Formatted at 2026-08-10 07:17:40 on dist-test-slave-gjrz"
server_key: "1be580634f709c709edbd740b5bfb758"
server_key_iv: "77107937d5520bd9875c7da5645e51ca"
server_key_version: "encryptionkey@0"
I20260810 07:17:42.980832 6465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:42.990895 6465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:42.993307 6465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:42.993440 6465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:42.993896 6465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:42.994644 6465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:42.994690 6465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:42.994725 6465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:42.994745 6465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:43.002949 6465 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:39919
I20260810 07:17:43.002980 6697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:39919 every 8 connection(s)
I20260810 07:17:43.003877 6465 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:17:43.004734 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 6465
I20260810 07:17:43.007704 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:43.006042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:51431)
0810 07:17:43.006483 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:43.006596 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":115,"threads_started":1}
W20260810 07:17:43.008112 6699 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:39829 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:39829: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
I20260810 07:17:43.008654 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:43.008468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:43563)
0810 07:17:43.008557 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:43.008603 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 07:17:43.009177 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:43.008998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:47263)
0810 07:17:43.009090 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:43.009130 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:17:43.009333 6699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:17:43.664543 6706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:43.664142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:40819)
0810 07:17:43.664396 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:43.664448 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":72,"threads_started":1}
I20260810 07:17:43.834090 6711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:43.833735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:41649)
0810 07:17:43.833942 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:43.833993 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":65,"threads_started":1}
I20260810 07:17:44.010138 6712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.009767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:55075)
0810 07:17:44.009989 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:44.010038 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260810 07:17:44.243746 6713 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:44.243999 6713 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:44.245075 6713 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919), 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
Aug 10 07:17:44 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.131@KRBTEST.COM
Aug 10 07:17:44 dist-test-slave-gjrz krb5kdc[3997](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346262, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.130@KRBTEST.COM
I20260810 07:17:44.263047 6715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.245252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:39919 (local address 127.12.117.129:45281)
0810 07:17:44.245568 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:44.245583 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:44.245689 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:44.247802 (+ 2113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:44.247808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:44.248430 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:44.249229 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:44.249252 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.251184 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.251188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:44.252929 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:44.252944 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.254409 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.254417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:44.254517 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:44.257269 (+ 2752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:44.257288 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:44.258792 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:44.260757 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:44.260764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:44.260778 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:44.261032 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:44.261363 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:44.261369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:44.261506 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:44.262133 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:44.262139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:44.262345 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 07:17:44.262516 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:17:44.262841 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":241,"spinlock_wait_cycles":5504,"thread_start_us":70,"threads_started":1}
I20260810 07:17:44.263069 6718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.245451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45281 (local address 127.12.117.131:39919)
0810 07:17:44.245868 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:44.245896 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:44.245942 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:17:44.246832 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:44.246841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:44.247330 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:44.247667 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:44.249418 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.250986 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.253052 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.253794 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.253985 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:44.259046 (+ 5061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:44.259052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:44.259066 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:44.259094 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:44.260634 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261150 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:44.261155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:44.261159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:44.261271 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261596 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:44.261599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:44.261600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:44.262011 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:44.262139 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:44.262648 (+ 509us) server_negotiation.cc:299] Negotiation successful
0810 07:17:44.262913 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":116,"threads_started":1}
I20260810 07:17:44.263084 6716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.245459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:53533 (local address 127.12.117.130:44393)
0810 07:17:44.245868 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:44.245896 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:44.245942 (+ 46us) server_negotiation.cc:407] Connection header received
0810 07:17:44.246832 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:44.246842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:44.247330 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:44.247667 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:44.249418 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.250986 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.253052 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.253794 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.253969 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:44.259319 (+ 5350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:44.259325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:44.259336 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:44.259365 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:44.260962 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261382 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:44.261388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:44.261394 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:44.261512 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261775 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:44.261778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:44.261780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:44.262167 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:44.262282 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:44.262648 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 07:17:44.262899 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":83,"threads_started":1}
I20260810 07:17:44.263077 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.245252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:44393 (local address 127.12.117.129:53533)
0810 07:17:44.245658 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:44.245669 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:44.245754 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:44.247802 (+ 2048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:44.247808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:44.248430 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:44.249229 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:44.249252 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.251193 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.251196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:44.252938 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:44.252944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:44.253916 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:44.253920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:44.254147 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:44.257264 (+ 3117us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:44.257282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:44.259102 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:44.261076 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:44.261082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:44.261276 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:44.261603 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:44.261606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:44.261616 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:44.261694 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:44.262272 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:44.262275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:44.262371 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 07:17:44.262515 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 07:17:44.262921 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":486,"spinlock_wait_cycles":8448,"thread_start_us":55,"threads_started":1}
W20260810 07:17:44.265780 6019 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
W20260810 07:17:44.265964 6020 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
I20260810 07:17:44.266042 6020 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20; no voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0
I20260810 07:17:44.266263 6713 raft_consensus.cc:2726] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:44.665431 6714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.665194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:53367)
0810 07:17:44.665300 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:44.665350 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:17:44.835095 6720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:44.834733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:35351)
0810 07:17:44.834958 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:44.835009 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260810 07:17:45.011137 6724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:45.010749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:38251)
0810 07:17:45.010998 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:45.011050 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
I20260810 07:17:45.666452 6727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:45.666049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:38311)
0810 07:17:45.666303 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:45.666357 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260810 07:17:45.836206 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:45.835782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:51275)
0810 07:17:45.836060 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:45.836121 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
I20260810 07:17:45.941620 6731 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:45.941748 6731 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:45.942121 6731 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919), 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
W20260810 07:17:45.942936 6020 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
W20260810 07:17:45.943060 6019 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
I20260810 07:17:45.943125 6019 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20; no voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0
I20260810 07:17:45.943252 6731 raft_consensus.cc:2726] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:46.012203 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:46.011801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:38501)
0810 07:17:46.012065 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:46.012118 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260810 07:17:46.667661 6734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:46.667162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:53691)
0810 07:17:46.667483 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:46.667551 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":96,"threads_started":1}
I20260810 07:17:46.837204 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:46.836840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:34017)
0810 07:17:46.837054 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:46.837111 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":69,"threads_started":1}
I20260810 07:17:47.013209 6736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:47.012827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:48749)
0810 07:17:47.013058 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:47.013117 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
I20260810 07:17:47.623606 6738 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:47.623750 6738 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:47.624137 6738 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919), 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
W20260810 07:17:47.624856 6019 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
W20260810 07:17:47.624994 6020 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
I20260810 07:17:47.625098 6020 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20; no voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0
I20260810 07:17:47.625250 6738 raft_consensus.cc:2726] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:47.668756 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:47.668381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:56507)
0810 07:17:47.668627 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:47.668674 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":65,"threads_started":1}
I20260810 07:17:47.838290 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:47.837902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:53711)
0810 07:17:47.838150 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:47.838203 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":73,"threads_started":1}
I20260810 07:17:48.014295 6743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:48.013876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:50647)
0810 07:17:48.014149 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:48.014204 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
I20260810 07:17:48.669742 6746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:48.669366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:48471)
0810 07:17:48.669594 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:48.669647 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":75,"threads_started":1}
I20260810 07:17:48.839365 6747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:48.838980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:43147)
0810 07:17:48.839200 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:48.839253 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":71,"threads_started":1}
I20260810 07:17:49.015388 6748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:49.014936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:52209)
0810 07:17:49.015191 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:49.015246 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":71,"threads_started":1}
I20260810 07:17:49.670763 6749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:49.670394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:50941)
0810 07:17:49.670621 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:49.670671 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260810 07:17:49.840446 6750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:49.840019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:36775)
0810 07:17:49.840303 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:49.840356 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260810 07:17:50.016408 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:50.015992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:45075)
0810 07:17:50.016269 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:50.016321 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
I20260810 07:17:50.671779 6755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:50.671422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:38591)
0810 07:17:50.671641 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:50.671689 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260810 07:17:50.841413 6757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:50.841052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:42955)
0810 07:17:50.841280 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:50.841330 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260810 07:17:50.992277 6758 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:50.992414 6758 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:50.992784 6758 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919), 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
W20260810 07:17:50.993485 6019 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
W20260810 07:17:50.993600 6020 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
I20260810 07:17:50.993669 6020 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20; no voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0
I20260810 07:17:50.993796 6758 raft_consensus.cc:2726] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:51.017364 6759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:51.016971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:47961)
0810 07:17:51.017224 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:51.017277 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260810 07:17:51.672780 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:51.672422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:40889)
0810 07:17:51.672638 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:51.672686 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":77,"threads_started":1}
I20260810 07:17:51.842447 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:51.842040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:59149)
0810 07:17:51.842292 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:51.842355 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
I20260810 07:17:52.018371 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:52.017979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:59799)
0810 07:17:52.018229 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:52.018280 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260810 07:17:52.673905 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:52.673520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:49331)
0810 07:17:52.673758 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:52.673810 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":71,"threads_started":1}
I20260810 07:17:52.843524 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:52.843103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:43357)
0810 07:17:52.843372 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:52.843430 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":83,"threads_started":1}
I20260810 07:17:53.019518 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:53.019083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:60729)
0810 07:17:53.019379 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:53.019430 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":97,"threads_started":1}
I20260810 07:17:53.674989 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:53.674580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:46877)
0810 07:17:53.674847 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:53.674897 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260810 07:17:53.844566 6773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:53.844173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:52243)
0810 07:17:53.844431 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:53.844482 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260810 07:17:54.020555 6776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:54.020164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:39179)
0810 07:17:54.020408 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:54.020462 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260810 07:17:54.676040 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:54.675659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:53841)
0810 07:17:54.675890 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:54.675943 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":72,"threads_started":1}
I20260810 07:17:54.845672 6779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:54.845250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:36995)
0810 07:17:54.845521 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:54.845576 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260810 07:17:55.021641 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:55.021246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:56651)
0810 07:17:55.021494 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:55.021545 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260810 07:17:55.234208 6781 raft_consensus.cc:482] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:17:55.234337 6781 raft_consensus.cc:504] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696bd1c5301e4638a086fac39cf822d0" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 39919 } } peers { permanent_uuid: "27aace958e024aec95a40bc13b35fc4f" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 44393 } } peers { permanent_uuid: "6769c5a05d414bbc97f1c7c468e25f20" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 38139 } }
I20260810 07:17:55.234748 6781 leader_election.cc:288] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919), 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393)
W20260810 07:17:55.235486 6019 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27aace958e024aec95a40bc13b35fc4f (127.12.117.130:44393): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
W20260810 07:17:55.235581 6020 leader_election.cc:341] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 696bd1c5301e4638a086fac39cf822d0 (127.12.117.131:39919): Not found: Tablet not found: f4284889905748dca6393b8d2eef3971
I20260810 07:17:55.235679 6020 leader_election.cc:302] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6769c5a05d414bbc97f1c7c468e25f20; no voters: 27aace958e024aec95a40bc13b35fc4f, 696bd1c5301e4638a086fac39cf822d0
I20260810 07:17:55.235837 6781 raft_consensus.cc:2726] T f4284889905748dca6393b8d2eef3971 P 6769c5a05d414bbc97f1c7c468e25f20 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:17:55.677016 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:55.676646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:38011)
0810 07:17:55.676873 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:55.676924 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":76,"threads_started":1}
I20260810 07:17:55.846782 6784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:55.846360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:45523)
0810 07:17:55.846638 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:55.846691 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260810 07:17:56.022740 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:56.022330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:60693)
0810 07:17:56.022597 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:56.022648 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":89,"threads_started":1}
I20260810 07:17:56.677983 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:56.677614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:48563)
0810 07:17:56.677839 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:56.677890 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
I20260810 07:17:56.847788 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:56.847414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:44095)
0810 07:17:56.847647 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:56.847699 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260810 07:17:57.023811 6797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:57.023419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:46573)
0810 07:17:57.023666 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:57.023719 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260810 07:17:57.679076 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:57.678713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.129:50797)
0810 07:17:57.678937 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:57.678987 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260810 07:17:57.848843 6800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:57.848498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.130:49041)
0810 07:17:57.848710 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:57.848757 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":69,"threads_started":1}
I20260810 07:17:58.015034 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 5936
I20260810 07:17:58.025030 6801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:58.024644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:39829 (local address 127.12.117.131:34555)
0810 07:17:58.024883 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:58.024937 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:39829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260810 07:17:58.028287 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 6213
I20260810 07:17:58.040185 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 6465
2026-08-10T07:17:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19454 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:17:58 dist-test-slave-gjrz krb5kdc[6805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:17:58 dist-test-slave-gjrz krb5kdc[6805](info): set up 2 sockets
Aug 10 07:17:58 dist-test-slave-gjrz krb5kdc[6805](info): commencing operation
krb5kdc: starting...
W20260810 07:17:59.205909 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:17:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:17:59.335630 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:59.439953 6830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:59.440172 6830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:59.440215 6830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:59.444090 6830 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:17:59.444131 6830 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:17:59.444145 6830 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:17:59.444170 6830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:17:59.444200 6830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:59.444229 6830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:59.444258 6830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:17:59.444288 6830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:17:59.449220 6830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:59.450093 6830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:59.450981 6830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:59.458659 6835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:59.458648 6839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:59.458680 6837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:59.458770 6830 server_base.cc:1034] running on GCE node
I20260810 07:17:59.459461 6830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:59.460428 6830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:59.461570 6830 hybrid_clock.cc:648] HybridClock initialized: now 1786346279461550 us; error 33 us; skew 500 ppm
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:59.464097 6830 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:17:59.464830 6830 webserver.cc:460] Webserver started at http://127.12.117.190:39121/ using document root <none> and password file <none>
I20260810 07:17:59.465312 6830 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:59.465375 6830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:59.465551 6830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:59.467425 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2d00d0303e0549c185c23d607c83d7bd"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "c509d5a10bb2f1579ee31c55dfd68dcc"
server_key_iv: "82c47d7c0ba2f1c6f1cbf691764e0fc1"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.467931 6830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2d00d0303e0549c185c23d607c83d7bd"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "c509d5a10bb2f1579ee31c55dfd68dcc"
server_key_iv: "82c47d7c0ba2f1c6f1cbf691764e0fc1"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.471335 6830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:17:59.473483 6851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.474556 6830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:59.474668 6830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2d00d0303e0549c185c23d607c83d7bd"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "c509d5a10bb2f1579ee31c55dfd68dcc"
server_key_iv: "82c47d7c0ba2f1c6f1cbf691764e0fc1"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.474753 6830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:59.484663 6830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:59.486943 6830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:59.487102 6830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:59.493901 6830 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:43387
I20260810 07:17:59.494223 6934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:43387 every 8 connection(s)
I20260810 07:17:59.494947 6830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:17:59.497948 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:17:59.500138 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 6830
I20260810 07:17:59.500252 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:17:59.500495 12758 external_mini_cluster.cc:1428] Setting key ef23ff8b2198db7db4c9367ff5fca7e6
I20260810 07:17:59.503957 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap starting.
I20260810 07:17:59.506438 6940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Neither blocks nor log segments found. Creating new log.
I20260810 07:17:59.507299 6940 log.cc:824] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:17:59.509248 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: No bootstrap required, opened a new log
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:17:59.513458 6940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:17:59.513728 6940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:17:59.513787 6940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Initialized, Role: FOLLOWER
I20260810 07:17:59.514237 6940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:17:59.514389 6940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:17:59.514452 6940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:17:59.514542 6940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:17:59.514654 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:59.501737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41248 (local address 127.12.117.190:43387)
0810 07:17:59.502159 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:59.502177 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:59.502218 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:17:59.502990 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:59.503014 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:59.503450 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:59.503797 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:59.504439 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.505869 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.506713 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.507426 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.507580 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:59.509934 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:59.509941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:59.509956 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:59.509985 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:59.511845 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.512238 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.512244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.512250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.512393 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.512622 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.512625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.512627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.513126 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:59.513312 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:59.513482 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 07:17:59.513766 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260810 07:17:59.515478 6940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:17:59.515851 6940 leader_election.cc:302] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d00d0303e0549c185c23d607c83d7bd; no voters:
I20260810 07:17:59.516184 6940 leader_election.cc:288] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:17:59.516264 6945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:17:59.516467 6945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 LEADER]: Becoming Leader. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Running, Role: LEADER
I20260810 07:17:59.516996 6945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:17:59.517338 6940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:17:59.518447 6946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:17:59.518580 6946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:59.518977 6961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:17:59.518930 6948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d00d0303e0549c185c23d607c83d7bd. Latest consensus state: current_term: 1 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:17:59.520054 6948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:17:59.521620 6961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:17:59.527163 6961 catalog_manager.cc:1347] Generated new cluster ID: ae91d31e91ce4e629dc9ca875b0b8ac7
I20260810 07:17:59.527225 6961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:17:59.537284 6961 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:17:59.538189 6961 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:17:59.543650 6961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Generated new TSK 0
I20260810 07:17:59.544341 6961 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:17:59.576092 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:59.683561 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:59.683789 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:59.683830 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:59.687736 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:59.687784 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:59.687837 6983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:17:59.692857 6983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:59.693714 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:59.694590 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:59.703429 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:59.703485 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:59.703454 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:59.703553 6983 server_base.cc:1034] running on GCE node
I20260810 07:17:59.704082 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:59.704658 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:59.705812 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786346279705792 us; error 33 us; skew 500 ppm
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:59.708415 6983 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:17:59.709209 6983 webserver.cc:460] Webserver started at http://127.12.117.129:39999/ using document root <none> and password file <none>
I20260810 07:17:59.709714 6983 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:59.709771 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:59.709931 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:59.711906 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "f4dfff004326b387c0d035c5df3e4328"
server_key_iv: "7808250b3d502ff2507f1fcbd978ab6b"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.712394 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "f4dfff004326b387c0d035c5df3e4328"
server_key_iv: "7808250b3d502ff2507f1fcbd978ab6b"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.715768 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:17:59.718058 7008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.719544 6983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:17:59.719717 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "f4dfff004326b387c0d035c5df3e4328"
server_key_iv: "7808250b3d502ff2507f1fcbd978ab6b"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.719828 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:59.746794 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:59.749361 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:59.749521 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:59.750038 6983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:59.750802 6983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:59.750850 6983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.750901 6983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:59.750927 6983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.758960 6983 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:35637
I20260810 07:17:59.758989 7210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:35637 every 8 connection(s)
I20260810 07:17:59.759800 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:17:59.761685 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 6983
I20260810 07:17:59.761777 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:17:59.762048 12758 external_mini_cluster.cc:1428] Setting key def5d52a690c99adeafa1feff5146902
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:17:59.773133 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:59.761362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:44115 (local address 127.12.117.190:43387)
0810 07:17:59.761510 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:59.761515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:59.762540 (+ 1025us) server_negotiation.cc:407] Connection header received
0810 07:17:59.763747 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:59.763751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:59.763803 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:59.763878 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:59.764972 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.765571 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.767002 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.767243 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.767355 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:59.769905 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:59.769908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:59.769910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:59.769936 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:59.771214 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.771804 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.771807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.771809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.771899 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.772230 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.772232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.772234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.772400 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:59.772470 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:59.772907 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 07:17:59.773012 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:17:59.773794 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:59.761730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:44115)
0810 07:17:59.762230 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:59.762281 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:59.763542 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:59.764028 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:59.764034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:59.764375 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:59.764822 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:59.764835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.765701 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.765705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:59.766876 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:59.766887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.767405 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.767409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:59.767609 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:59.768196 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:59.768213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:59.769736 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:59.771374 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:59.771380 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:59.771395 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:59.771713 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:59.771972 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:59.771976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:59.771978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:59.772159 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:59.772473 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:59.772478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:59.772699 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:17:59.772853 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 07:17:59.773099 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:17:59.774991 7211 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:17:59.775310 7211 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:59.775923 7211 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:17:59.777544 6875 ts_manager.cc:194] Registered new tserver with Master: 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637)
I20260810 07:17:59.778707 6875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:44115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:17:59.787791 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:17:59.894405 7219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:17:59.894621 7219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:17:59.894667 7219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:17:59.898579 7219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:17:59.898630 7219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:17:59.898699 7219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:17:59.903755 7219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:17:59.904619 7219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:17:59.905514 7219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:17:59.914322 7227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:17:59.914324 7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:59.914462 7219 server_base.cc:1034] running on GCE node
W20260810 07:17:59.914397 7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:17:59.914975 7219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:17:59.915604 7219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:17:59.916791 7219 hybrid_clock.cc:648] HybridClock initialized: now 1786346279916774 us; error 25 us; skew 500 ppm
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:17:59.919549 7219 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:17:59.920303 7219 webserver.cc:460] Webserver started at http://127.12.117.130:33987/ using document root <none> and password file <none>
I20260810 07:17:59.920784 7219 fs_manager.cc:359] Metadata directory not provided
I20260810 07:17:59.920843 7219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:17:59.921028 7219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:17:59.922905 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "f9d9bada3fc04730a6825221f76d17f0"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "ccb9cca669663ecee773bbc5e315a54a"
server_key_iv: "5215e4794acf87fa1b0a08cafc6fee22"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.923454 7219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "f9d9bada3fc04730a6825221f76d17f0"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "ccb9cca669663ecee773bbc5e315a54a"
server_key_iv: "5215e4794acf87fa1b0a08cafc6fee22"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.926796 7219 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:17:59.929023 7239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.930147 7219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:17:59.930243 7219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "f9d9bada3fc04730a6825221f76d17f0"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "ccb9cca669663ecee773bbc5e315a54a"
server_key_iv: "5215e4794acf87fa1b0a08cafc6fee22"
server_key_version: "encryptionkey@0"
I20260810 07:17:59.930341 7219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:17:59.968569 7219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:17:59.971487 7219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:17:59.971649 7219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:17:59.972184 7219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:17:59.972934 7219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:17:59.972992 7219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.973043 7219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:17:59.973080 7219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:17:59.981273 7219 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:38913
I20260810 07:17:59.981302 7439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:38913 every 8 connection(s)
I20260810 07:17:59.982069 7219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:17:59.983296 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 7219
I20260810 07:17:59.983423 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:17:59.983747 12758 external_mini_cluster.cc:1428] Setting key e693e68c434c14e4cd5991efc93f8f60
Aug 10 07:17:59 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346279, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:17:59.995769 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:59.983766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:57037 (local address 127.12.117.190:43387)
0810 07:17:59.983886 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:17:59.983890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:17:59.984817 (+ 927us) server_negotiation.cc:407] Connection header received
0810 07:17:59.986356 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:17:59.986358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:17:59.986404 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:17:59.986470 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:17:59.987797 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.988566 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.989644 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.990000 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.990128 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:59.992522 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:17:59.992526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:17:59.992529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:17:59.992562 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:17:59.994009 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.994487 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.994491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.994493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.994590 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:17:59.994850 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:17:59.994852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:17:59.994854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:17:59.995025 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:17:59.995099 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:17:59.995547 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 07:17:59.995652 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:17:59.996479 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:17:59.984209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:57037)
0810 07:17:59.984664 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 07:17:59.984713 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:17:59.986131 (+ 1418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:17:59.986664 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:17:59.986672 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:17:59.987113 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:17:59.987652 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:59.987667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.988672 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.988675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:59.989519 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:17:59.989528 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:17:59.990150 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:17:59.990155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:17:59.990304 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:17:59.990814 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:17:59.990830 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:17:59.992349 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:17:59.994118 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:59.994122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:59.994134 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:59.994393 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:59.994667 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:17:59.994669 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:17:59.994670 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:17:59.994785 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:17:59.995097 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:17:59.995101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:17:59.995343 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 07:17:59.995491 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 07:17:59.995727 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:17:59.997618 7441 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:17:59.997838 7441 heartbeater.cc:464] Registering TS with master...
I20260810 07:17:59.998275 7441 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:17:59.999372 6875 ts_manager.cc:194] Registered new tserver with Master: f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913)
I20260810 07:17:59.999975 6875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:57037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:18:00.011407 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:00.120142 7447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:00.120357 7447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:00.120399 7447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:00.124365 7447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:00.124414 7447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:00.124478 7447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:18:00.129464 7447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:00.130347 7447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:00.131232 7447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:00.140259 7452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:00.140257 7453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:00.140347 7447 server_base.cc:1034] running on GCE node
W20260810 07:18:00.140524 7461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:00.140868 7447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:00.141451 7447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:00.142617 7447 hybrid_clock.cc:648] HybridClock initialized: now 1786346280142597 us; error 27 us; skew 500 ppm
Aug 10 07:18:00 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346280, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:00.145200 7447 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:18:00.146023 7447 webserver.cc:460] Webserver started at http://127.12.117.131:40571/ using document root <none> and password file <none>
I20260810 07:18:00.146557 7447 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:00.146624 7447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:00.146792 7447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:00.148748 7447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "f3179b78a91c4625b538722eacc9e726"
format_stamp: "Formatted at 2026-08-10 07:18:00 on dist-test-slave-gjrz"
server_key: "003da725465ac7adc9203979cc6f4e90"
server_key_iv: "b75ceb4b24ad7c0737a25039d9633237"
server_key_version: "encryptionkey@0"
I20260810 07:18:00.149245 7447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "f3179b78a91c4625b538722eacc9e726"
format_stamp: "Formatted at 2026-08-10 07:18:00 on dist-test-slave-gjrz"
server_key: "003da725465ac7adc9203979cc6f4e90"
server_key_iv: "b75ceb4b24ad7c0737a25039d9633237"
server_key_version: "encryptionkey@0"
I20260810 07:18:00.152701 7447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 07:18:00.154901 7472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:00.156064 7447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:00.156162 7447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "f3179b78a91c4625b538722eacc9e726"
format_stamp: "Formatted at 2026-08-10 07:18:00 on dist-test-slave-gjrz"
server_key: "003da725465ac7adc9203979cc6f4e90"
server_key_iv: "b75ceb4b24ad7c0737a25039d9633237"
server_key_version: "encryptionkey@0"
I20260810 07:18:00.156244 7447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:00.177518 7447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:00.180424 7447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:00.180565 7447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:00.181056 7447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:00.181784 7447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:00.181831 7447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:00.181866 7447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:00.181901 7447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:00.189841 7447 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41117
I20260810 07:18:00.189893 7692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41117 every 8 connection(s)
I20260810 07:18:00.190645 7447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:00.197296 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 7447
I20260810 07:18:00.197396 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:00.197643 12758 external_mini_cluster.cc:1428] Setting key 2a178d0f6c70ed87e30a1353e64564ba
Aug 10 07:18:00 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346280, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:00.203095 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.192277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:43033 (local address 127.12.117.190:43387)
0810 07:18:00.192405 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.192409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.193151 (+ 742us) server_negotiation.cc:407] Connection header received
0810 07:18:00.194440 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.194443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.194502 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.194573 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:00.195737 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.196264 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.197408 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.197661 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.197753 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.200377 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:00.200380 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:00.200382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:00.200406 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:00.201509 (+ 1103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:00.201951 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:00.201953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:00.201954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:00.202018 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:00.202264 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:00.202266 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:00.202267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:00.202420 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:00.202488 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.202889 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.202985 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:00.203904 7698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.192598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:43033)
0810 07:18:00.193011 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.193051 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.194233 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.194752 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.194759 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.195088 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:00.195593 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.195608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.196390 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.196393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.197283 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.197294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.197897 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.197901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.198088 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.198634 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:00.198651 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:00.200228 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:00.201605 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:00.201616 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:00.201627 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:00.201865 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:00.202077 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:00.202079 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:00.202080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:00.202201 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:00.202488 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:00.202493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:00.202712 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.202858 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.203086 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":109,"threads_started":1}
I20260810 07:18:00.204924 7694 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:18:00.205132 7694 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:00.205554 7694 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:18:00.206475 6875 ts_manager.cc:194] Registered new tserver with Master: f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
I20260810 07:18:00.207098 6875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:43033
I20260810 07:18:00.211184 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:00.219193 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.212986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.12.117.190:43387)
0810 07:18:00.213124 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.213128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.213260 (+ 132us) server_negotiation.cc:407] Connection header received
0810 07:18:00.213351 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.213353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.213388 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.213439 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:00.214035 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.214512 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.215249 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.215480 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.215550 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.216702 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:00.216705 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:00.216706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:00.216720 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:00.217816 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:00.218206 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:00.218208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:00.218210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:00.218271 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:00.218452 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:00.218454 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:00.218455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:00.218672 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:00.218771 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.218958 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.219043 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:18:00.221598 6875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:00.224674 6875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:00.238801 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.233353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:35637 (local address 127.0.0.1:49380)
0810 07:18:00.233946 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.233989 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.234126 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.234631 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.234638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.234676 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:00.234822 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.234828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.236292 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.236296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.237324 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.237334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.238254 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.238258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.238376 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.238420 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.238530 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.238650 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":367,"spinlock_wait_cycles":3712,"thread_start_us":78,"threads_started":1}
I20260810 07:18:00.238833 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.233496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.12.117.129:35637)
0810 07:18:00.233848 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.233862 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.234262 (+ 400us) server_negotiation.cc:407] Connection header received
0810 07:18:00.234342 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.234347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.234495 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.234631 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:00.234926 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.236145 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.237446 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.238139 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.238257 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.238404 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.238580 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.238681 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":80,"threads_started":1}
I20260810 07:18:00.239362 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.233281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.12.117.131:41117)
0810 07:18:00.233709 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.233725 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.234145 (+ 420us) server_negotiation.cc:407] Connection header received
0810 07:18:00.234787 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.234792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.234947 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.235084 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:00.235223 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.236310 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.237798 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.238566 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.238676 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.238790 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.239113 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.239211 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":140,"threads_started":1}
I20260810 07:18:00.239720 7721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.232666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41117 (local address 127.0.0.1:36584)
0810 07:18:00.233935 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.233971 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.234128 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.235066 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.235068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.235083 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:00.235156 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.235161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.236421 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.236425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.237666 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.237679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.238856 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.238860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.238947 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.238969 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.239537 (+ 568us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.239608 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":36,"thread_start_us":107,"threads_started":1}
I20260810 07:18:00.240746 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.232549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38913 (local address 127.0.0.1:49422)
0810 07:18:00.233946 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.233989 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.234138 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.234631 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.234637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.234675 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:00.236633 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.236641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.238012 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.238015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.239025 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.239037 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.240357 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.240361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.240459 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.240489 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.240559 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.240623 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1155,"mutex_wait_us":1798,"spinlock_wait_cycles":7296,"thread_start_us":66,"threads_started":1}
I20260810 07:18:00.240860 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.232692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49422 (local address 127.12.117.130:38913)
0810 07:18:00.233645 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.233658 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.234075 (+ 417us) server_negotiation.cc:407] Connection header received
0810 07:18:00.234282 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.234289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.234481 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.234617 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:00.236762 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.237896 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.239209 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.240027 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.240136 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.240258 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.240623 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.240719 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":129,"threads_started":1}
I20260810 07:18:00.242336 7097 tablet_service.cc:1467] Processing CreateTablet for tablet 30f4dc19482b4e70a2db3fac494c470e (DEFAULT_TABLE table=test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:00.243400 7097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f4dc19482b4e70a2db3fac494c470e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:00.243165 7328 tablet_service.cc:1467] Processing CreateTablet for tablet 30f4dc19482b4e70a2db3fac494c470e (DEFAULT_TABLE table=test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:00.244113 7328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f4dc19482b4e70a2db3fac494c470e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:00.249235 7728 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap starting.
I20260810 07:18:00.249637 7575 tablet_service.cc:1467] Processing CreateTablet for tablet 30f4dc19482b4e70a2db3fac494c470e (DEFAULT_TABLE table=test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:00.250561 7575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30f4dc19482b4e70a2db3fac494c470e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:00.251221 7728 tablet_bootstrap.cc:654] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:00.251928 7729 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Bootstrap starting.
I20260810 07:18:00.252509 7728 log.cc:824] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:00.254328 7729 tablet_bootstrap.cc:654] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:00.254640 7728 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: No bootstrap required, opened a new log
I20260810 07:18:00.254842 7728 ts_tablet_manager.cc:1397] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:18:00.255007 7729 log.cc:824] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:00.256634 7729 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: No bootstrap required, opened a new log
I20260810 07:18:00.256800 7733 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Bootstrap starting.
I20260810 07:18:00.256867 7729 ts_tablet_manager.cc:1397] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260810 07:18:00.258325 7728 raft_consensus.cc:348] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.258535 7728 raft_consensus.cc:374] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:00.258625 7728 raft_consensus.cc:729] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390b49248b7942bfaf2fa795cfbf3f9a, State: Initialized, Role: FOLLOWER
I20260810 07:18:00.258875 7733 tablet_bootstrap.cc:654] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:00.259058 7728 consensus_queue.cc:260] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.259708 7211 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
I20260810 07:18:00.260036 7728 ts_tablet_manager.cc:1428] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260810 07:18:00.260690 7733 log.cc:824] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:00.260655 7729 raft_consensus.cc:348] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.260903 7729 raft_consensus.cc:374] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260810 07:18:00.260901 7212 tablet.cc:2369] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:00.260952 7729 raft_consensus.cc:729] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d9bada3fc04730a6825221f76d17f0, State: Initialized, Role: FOLLOWER
I20260810 07:18:00.261375 7729 consensus_queue.cc:260] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.262074 7441 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
I20260810 07:18:00.262392 7729 ts_tablet_manager.cc:1428] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:18:00.262578 7733 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: No bootstrap required, opened a new log
I20260810 07:18:00.262737 7733 ts_tablet_manager.cc:1397] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:18:00.265735 7733 raft_consensus.cc:348] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.265919 7733 raft_consensus.cc:374] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:00.265951 7733 raft_consensus.cc:729] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3179b78a91c4625b538722eacc9e726, State: Initialized, Role: FOLLOWER
I20260810 07:18:00.266271 7733 consensus_queue.cc:260] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.266788 7694 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
I20260810 07:18:00.267030 7733 ts_tablet_manager.cc:1428] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260810 07:18:00.267199 7734 raft_consensus.cc:482] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:00.267361 7734 raft_consensus.cc:504] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.268254 7734 leader_election.cc:288] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
I20260810 07:18:00.272243 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.268448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38913 (local address 127.12.117.129:41711)
0810 07:18:00.268683 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.268696 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.268763 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.269316 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.269319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.269337 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:00.269466 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.269472 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.270248 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.270251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.271060 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.271069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.271849 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.271853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.271956 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.271979 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.272044 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.272115 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":6,"spinlock_wait_cycles":12288}
I20260810 07:18:00.272239 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.268908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:41711 (local address 127.12.117.130:38913)
0810 07:18:00.269027 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.269031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.269043 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:18:00.269082 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.269085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.269127 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.269196 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:00.269603 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.270146 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.271173 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.271749 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.271849 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.271900 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.272066 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.272138 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:18:00.272606 7742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.268448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41117 (local address 127.12.117.129:36399)
0810 07:18:00.269251 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.269264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:00.269385 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:00.269578 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:00.269580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:00.269595 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:00.269708 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.269713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.270387 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.270391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.271495 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:00.271506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.272210 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.272214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:00.272313 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.272337 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:18:00.272408 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:18:00.272468 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":538,"spinlock_wait_cycles":2304,"thread_start_us":101,"threads_started":1}
I20260810 07:18:00.272621 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.268581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:36399 (local address 127.12.117.131:41117)
0810 07:18:00.268714 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.268719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.269359 (+ 640us) server_negotiation.cc:407] Connection header received
0810 07:18:00.269461 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.269464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.269503 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.269562 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:00.269813 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.270286 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.271620 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.272102 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.272205 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.272249 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.272433 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.272510 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 07:18:00.272840 7360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f4dc19482b4e70a2db3fac494c470e" candidate_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d9bada3fc04730a6825221f76d17f0" is_pre_election: true
I20260810 07:18:00.273064 7360 raft_consensus.cc:2454] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 390b49248b7942bfaf2fa795cfbf3f9a in term 0.
I20260810 07:18:00.273113 7611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f4dc19482b4e70a2db3fac494c470e" candidate_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3179b78a91c4625b538722eacc9e726" is_pre_election: true
I20260810 07:18:00.273314 7611 raft_consensus.cc:2454] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 390b49248b7942bfaf2fa795cfbf3f9a in term 0.
I20260810 07:18:00.273535 7014 leader_election.cc:302] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a, f9d9bada3fc04730a6825221f76d17f0; no voters:
I20260810 07:18:00.273792 7734 raft_consensus.cc:2781] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:00.273862 7734 raft_consensus.cc:482] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:00.273911 7734 raft_consensus.cc:3037] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:00.274787 7734 raft_consensus.cc:504] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.275074 7734 leader_election.cc:288] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 1 election: Requested vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
I20260810 07:18:00.275390 7611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f4dc19482b4e70a2db3fac494c470e" candidate_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3179b78a91c4625b538722eacc9e726"
I20260810 07:18:00.275390 7360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30f4dc19482b4e70a2db3fac494c470e" candidate_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9d9bada3fc04730a6825221f76d17f0"
I20260810 07:18:00.275497 7611 raft_consensus.cc:3037] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:00.275497 7360 raft_consensus.cc:3037] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:00.276566 7360 raft_consensus.cc:2454] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 390b49248b7942bfaf2fa795cfbf3f9a in term 1.
I20260810 07:18:00.276566 7611 raft_consensus.cc:2454] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 390b49248b7942bfaf2fa795cfbf3f9a in term 1.
I20260810 07:18:00.276854 7013 leader_election.cc:302] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a, f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0; no voters:
I20260810 07:18:00.277043 7734 raft_consensus.cc:2781] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:00.277192 7734 raft_consensus.cc:686] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 LEADER]: Becoming Leader. State: Replica: 390b49248b7942bfaf2fa795cfbf3f9a, State: Running, Role: LEADER
I20260810 07:18:00.277434 7734 consensus_queue.cc:237] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:00.280051 6874 catalog_manager.cc:5495] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129). New cstate: current_term: 1 leader_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } health_report { overall_health: HEALTHY } } }
I20260810 07:18:00.297816 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.294988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.12.117.129:35637)
0810 07:18:00.295093 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:00.295096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:00.295116 (+ 20us) server_negotiation.cc:407] Connection header received
0810 07:18:00.295236 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:00.295238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:00.295307 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:00.295386 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:00.295568 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.296021 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.296766 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:00.296960 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:00.297024 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:00.297234 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:00.297572 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:00.297654 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:00.297684 (+ 30us) server_negotiation.cc:299] Negotiation successful
0810 07:18:00.297726 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:18:00.302561 7611 raft_consensus.cc:1261] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Refusing update from remote peer 390b49248b7942bfaf2fa795cfbf3f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:00.302577 7360 raft_consensus.cc:1261] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Refusing update from remote peer 390b49248b7942bfaf2fa795cfbf3f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:00.303153 7743 consensus_queue.cc:1035] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:00.303331 7734 consensus_queue.cc:1035] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:00.319609 6875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:00.319768 6875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:00.328405 6875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Sending DeleteTablet for 3 replicas of tablet 30f4dc19482b4e70a2db3fac494c470e
I20260810 07:18:00.329180 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 6983
I20260810 07:18:00.329217 7328 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f4dc19482b4e70a2db3fac494c470e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:00 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:49422
I20260810 07:18:00.330704 7575 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f4dc19482b4e70a2db3fac494c470e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:00 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:36584
I20260810 07:18:00.333915 7766 tablet_replica.cc:330] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: stopping tablet replica
I20260810 07:18:00.333948 7765 tablet_replica.cc:330] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: stopping tablet replica
I20260810 07:18:00.334234 7766 raft_consensus.cc:2229] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:00.334260 7765 raft_consensus.cc:2229] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:00.334561 7765 raft_consensus.cc:2258] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:00.334558 7766 raft_consensus.cc:2258] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:00.336117 7766 ts_tablet_manager.cc:1905] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:00.337585 7765 ts_tablet_manager.cc:1905] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:00.340031 7765 ts_tablet_manager.cc:1918] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:00.340121 7765 log.cc:1196] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/30f4dc19482b4e70a2db3fac494c470e
I20260810 07:18:00.340245 7766 ts_tablet_manager.cc:1918] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:00.340348 7766 log.cc:1196] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/30f4dc19482b4e70a2db3fac494c470e
I20260810 07:18:00.340427 7765 ts_tablet_manager.cc:1939] T 30f4dc19482b4e70a2db3fac494c470e P f9d9bada3fc04730a6825221f76d17f0: Deleting consensus metadata
I20260810 07:18:00.340638 7766 ts_tablet_manager.cc:1939] T 30f4dc19482b4e70a2db3fac494c470e P f3179b78a91c4625b538722eacc9e726: Deleting consensus metadata
I20260810 07:18:00.341467 6854 catalog_manager.cc:4841] TS f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): tablet 30f4dc19482b4e70a2db3fac494c470e (table test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]) successfully deleted
I20260810 07:18:00.341668 6852 catalog_manager.cc:4841] TS f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): tablet 30f4dc19482b4e70a2db3fac494c470e (table test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]) successfully deleted
W20260810 07:18:00.344313 6852 connection.cc:535] client connection to 127.12.117.129:35637 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 07:18:00.344427 6852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:18:00.344570 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 7219
I20260810 07:18:00.350778 7719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:00.350493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:35637 (local address 127.0.0.1:49406)
0810 07:18:00.350636 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:00.350705 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:35637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 07:18:00.350926 6852 catalog_manager.cc:4584] TS 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30f4dc19482b4e70a2db3fac494c470e: Network error: Client connection negotiation failed: client connection to 127.12.117.129:35637: connect: Connection refused (error 111)
I20260810 07:18:00.355768 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 7447
I20260810 07:18:00.370822 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 6830
I20260810 07:18:20.381626 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--webserver_interface=127.12.117.190
--webserver_port=39121
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:20.495208 7768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:20.495478 7768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:20.495517 7768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:20.499404 7768 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:18:20.499446 7768 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:18:20.499461 7768 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:18:20.499473 7768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:20.499490 7768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:20.499505 7768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:20.499534 7768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:20.499560 7768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:20.504618 7768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:20.505527 7768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:20.506438 7768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:20.519621 7784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:20.519650 7788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:20.519706 7768 server_base.cc:1034] running on GCE node
W20260810 07:18:20.519645 7783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:20.520349 7768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:20.521376 7768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:20.522534 7768 hybrid_clock.cc:648] HybridClock initialized: now 1786346300522514 us; error 30 us; skew 500 ppm
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:20.525292 7768 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:18:20.526158 7768 webserver.cc:460] Webserver started at http://127.12.117.190:39121/ using document root <none> and password file <none>
I20260810 07:18:20.526698 7768 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:20.526774 7768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:20.530839 7768 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:18:20.533099 7798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:20.534265 7768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:18:20.534387 7768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2d00d0303e0549c185c23d607c83d7bd"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "c509d5a10bb2f1579ee31c55dfd68dcc"
server_key_iv: "82c47d7c0ba2f1c6f1cbf691764e0fc1"
server_key_version: "encryptionkey@0"
I20260810 07:18:20.534973 7768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:20.552577 7768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:20.555150 7768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:20.555343 7768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:20.562175 7768 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:43387
I20260810 07:18:20.562410 7889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:43387 every 8 connection(s)
I20260810 07:18:20.563234 7768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:20.567461 7890 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:18:20.569095 7890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap starting.
I20260810 07:18:20.569621 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 7768
I20260810 07:18:20.570035 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:35637
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=39999
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:20.586038 7890 log.cc:824] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:20.594465 7890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:20.594813 7890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap complete.
I20260810 07:18:20.598578 7890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:20.598942 7890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Initialized, Role: FOLLOWER
I20260810 07:18:20.599447 7890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:20.599568 7890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:20.599638 7890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:20.599730 7890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 1 FOLLOWER]: Advancing to term 2
I20260810 07:18:20.601627 7890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:20.602010 7890 leader_election.cc:302] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d00d0303e0549c185c23d607c83d7bd; no voters:
I20260810 07:18:20.602324 7890 leader_election.cc:288] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 07:18:20.602440 7896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Leader election won for term 2
I20260810 07:18:20.602699 7896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 LEADER]: Becoming Leader. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Running, Role: LEADER
I20260810 07:18:20.603004 7896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:20.603458 7890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:20.604388 7897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:18:20.604521 7897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:20.605113 7898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d00d0303e0549c185c23d607c83d7bd. Latest consensus state: current_term: 2 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:18:20.605209 7898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:20.605294 7911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:20.609688 7911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:20.610297 7911 catalog_manager.cc:1259] Loaded cluster ID: ae91d31e91ce4e629dc9ca875b0b8ac7
I20260810 07:18:20.610358 7911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:20.612833 7911 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:20.613516 7911 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Loaded TSK: 0
I20260810 07:18:20.630532 7911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Generated new TSK 1
I20260810 07:18:20.630771 7911 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:18:20.680208 7893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:20.680426 7893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:20.680485 7893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:20.684368 7893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:20.684417 7893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:20.684494 7893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:18:20.689364 7893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:35637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=39999
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:20.690219 7893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:20.691094 7893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:20.701078 7893 server_base.cc:1034] running on GCE node
W20260810 07:18:20.701033 7941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:20.701202 7942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:20.701035 7944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:20.701704 7893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:20.702286 7893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:20.703450 7893 hybrid_clock.cc:648] HybridClock initialized: now 1786346300703430 us; error 38 us; skew 500 ppm
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:20.706161 7893 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:18:20.706964 7893 webserver.cc:460] Webserver started at http://127.12.117.129:39999/ using document root <none> and password file <none>
I20260810 07:18:20.707499 7893 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:20.707571 7893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:20.711238 7893 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:18:20.713397 7957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:20.714531 7893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:20.714622 7893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "f4dfff004326b387c0d035c5df3e4328"
server_key_iv: "7808250b3d502ff2507f1fcbd978ab6b"
server_key_version: "encryptionkey@0"
I20260810 07:18:20.715119 7893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:20.743057 7893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:20.745774 7893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:20.745936 7893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:20.746470 7893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:20.747473 7970 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:18:20.751245 7893 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:18:20.751314 7893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260810 07:18:20.751376 7893 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:18:20.752923 7893 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:18:20.752978 7893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 07:18:20.753230 7970 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap starting.
I20260810 07:18:20.762203 7893 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:35637
I20260810 07:18:20.762279 8165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:35637 every 8 connection(s)
I20260810 07:18:20.763079 7893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:20.767073 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 7893
I20260810 07:18:20.767628 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:38913
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=33987
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:20.772172 7970 log.cc:824] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:20.779917 7970 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:20.780344 7970 tablet_bootstrap.cc:492] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap complete.
I20260810 07:18:20.781586 7970 ts_tablet_manager.cc:1397] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.005s
I20260810 07:18:20.781638 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.765064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:34139)
0810 07:18:20.765538 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:20.765578 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:20.766327 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:20.767659 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:20.767669 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:20.768408 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:20.769103 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.769124 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.771136 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.771139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.772523 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.772536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.773431 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.773435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.773635 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.774164 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:20.774180 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:20.775855 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:20.778345 (+ 2490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.778351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.778362 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.778694 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.779099 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.779102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.779104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.779288 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.779986 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:20.779992 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:20.780272 (+ 280us) client_negotiation.cc:769] Sending connection context
0810 07:18:20.780469 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 07:18:20.780764 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":145,"threads_started":1}
I20260810 07:18:20.783295 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.765043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:34139 (local address 127.12.117.190:43387)
0810 07:18:20.765379 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:20.765399 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:20.765687 (+ 288us) server_negotiation.cc:407] Connection header received
0810 07:18:20.766735 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:20.766746 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:20.767151 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:20.767537 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:20.769310 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.770961 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.772674 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.773291 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.773454 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.776103 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:20.776110 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:20.776127 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:20.776161 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:20.778212 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.778819 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.778826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.778832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.778993 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.779379 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.779382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.779384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.779854 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:20.780094 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:20.782194 (+ 2100us) server_negotiation.cc:299] Negotiation successful
0810 07:18:20.782499 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":102,"threads_started":1}
I20260810 07:18:20.784391 7970 raft_consensus.cc:348] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:20.784771 7970 raft_consensus.cc:729] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390b49248b7942bfaf2fa795cfbf3f9a, State: Initialized, Role: FOLLOWER
I20260810 07:18:20.785223 7970 consensus_queue.cc:260] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:20.785527 8166 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:18:20.785763 8166 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:20.786083 7970 ts_tablet_manager.cc:1428] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260810 07:18:20.786319 8166 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:18:20.788671 7829 ts_manager.cc:194] Registered new tserver with Master: 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637)
I20260810 07:18:20.793193 7829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:34139
I20260810 07:18:20.795817 8166 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.129@KRBTEST.COM
I20260810 07:18:20.802575 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.791677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:35637 (local address 127.0.0.1:59258)
0810 07:18:20.792297 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:20.792342 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:20.792470 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:20.794270 (+ 1800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:20.794275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:20.794606 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:20.794700 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.794707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.795682 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.795685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.796452 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.796464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.797184 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.797187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.797292 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.797762 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:20.797773 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:20.799159 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:20.801075 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.801084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.801087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.801299 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.801573 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.801576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.801577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.801646 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.802116 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:20.802120 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:20.802209 (+ 89us) client_negotiation.cc:769] Sending connection context
0810 07:18:20.802311 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:18:20.802459 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":98,"threads_started":1}
I20260810 07:18:20.802785 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.792289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.12.117.129:35637)
0810 07:18:20.793902 (+ 1613us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:20.793913 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:20.793932 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:18:20.793984 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:20.793988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:20.794125 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:20.794240 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:20.794819 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.795568 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.796574 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.797068 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.797170 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.799325 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:20.799335 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:20.799348 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:20.799372 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:20.800954 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.801398 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.801403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.801405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.801496 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.801718 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.801721 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.801723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.802012 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:20.802151 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:20.802401 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 07:18:20.802555 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1538,"thread_start_us":77,"threads_started":1}
I20260810 07:18:20.803666 8042 tablet_service.cc:1514] Processing DeleteTablet for tablet 30f4dc19482b4e70a2db3fac494c470e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:00 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59258
I20260810 07:18:20.804437 8185 tablet_replica.cc:330] stopping tablet replica
I20260810 07:18:20.804706 8185 raft_consensus.cc:2229] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:20.804940 8185 raft_consensus.cc:2258] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:20.805835 8185 ts_tablet_manager.cc:1905] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:20.809530 8185 ts_tablet_manager.cc:1918] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:20.809610 8185 log.cc:1196] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal/wals/30f4dc19482b4e70a2db3fac494c470e
I20260810 07:18:20.809834 8185 ts_tablet_manager.cc:1939] T 30f4dc19482b4e70a2db3fac494c470e P 390b49248b7942bfaf2fa795cfbf3f9a: Deleting consensus metadata
I20260810 07:18:20.811388 7801 catalog_manager.cc:4841] TS 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637): tablet 30f4dc19482b4e70a2db3fac494c470e (table test-table [id=c471b9f07d7e4b8bb1d4497c8b783611]) successfully deleted
W20260810 07:18:20.884677 8173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:20.884891 8173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:20.884944 8173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:20.888767 8173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:20.888815 8173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:20.888888 8173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:18:20.893846 8173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=33987
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:20.894698 8173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:20.895586 8173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:20.904618 8199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:20.904600 8196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:20.904593 8193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:20.904651 8173 server_base.cc:1034] running on GCE node
I20260810 07:18:20.905228 8173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:20.905799 8173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:20.906962 8173 hybrid_clock.cc:648] HybridClock initialized: now 1786346300906937 us; error 36 us; skew 500 ppm
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:20.909551 8173 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:18:20.910341 8173 webserver.cc:460] Webserver started at http://127.12.117.130:33987/ using document root <none> and password file <none>
I20260810 07:18:20.910828 8173 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:20.910902 8173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:20.914525 8173 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:20.916507 8208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:20.917639 8173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:20.917733 8173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "f9d9bada3fc04730a6825221f76d17f0"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "ccb9cca669663ecee773bbc5e315a54a"
server_key_iv: "5215e4794acf87fa1b0a08cafc6fee22"
server_key_version: "encryptionkey@0"
I20260810 07:18:20.918200 8173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:20.940284 8173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:20.942957 8173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:20.943118 8173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:20.943682 8173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:20.944443 8173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:20.944495 8173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:20.944556 8173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:20.944587 8173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:20.953078 8173 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:38913
I20260810 07:18:20.953110 8417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:38913 every 8 connection(s)
I20260810 07:18:20.953979 8173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:20.954874 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 8173
I20260810 07:18:20.955391 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:41117
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=40571
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 07:18:20 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:20.967691 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.957033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:58715 (local address 127.12.117.190:43387)
0810 07:18:20.957185 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:20.957191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:20.957209 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:18:20.958106 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:20.958110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:20.958180 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:20.958277 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:20.959366 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.960169 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.961280 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.961646 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.961757 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.964208 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:20.964212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:20.964215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:20.964246 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:20.965747 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.966218 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.966222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.966224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.966324 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:20.966608 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:20.966611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:20.966613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:20.966842 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:20.966946 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:20.967386 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 07:18:20.967532 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:18:20.968327 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:20.956074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:58715)
0810 07:18:20.956531 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:20.956581 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:20.957903 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:20.958414 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:20.958420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:20.958749 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:20.959201 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.959215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.960296 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.960298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.961160 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:20.961170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:20.961784 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:20.961787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:20.961941 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:20.962471 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:20.962488 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:20.964029 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:20.965857 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.965862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.965872 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.966120 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.966405 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:20.966407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:20.966408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:20.966523 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:20.966929 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:20.966933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:20.967152 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:18:20.967320 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 07:18:20.967558 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260810 07:18:20.969406 8420 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:18:20.969651 8420 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:20.970160 8420 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:18:20.971185 7829 ts_manager.cc:194] Registered new tserver with Master: f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913)
I20260810 07:18:20.971768 7829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:58715
W20260810 07:18:21.066547 8424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:21.066762 8424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:21.066802 8424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:21.070632 8424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:21.070679 8424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:21.070744 8424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:18:21.075631 8424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:41117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=40571
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:21.076501 8424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:21.077387 8424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:21.086299 8435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:21.086282 8436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:21.086285 8439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:21.086442 8424 server_base.cc:1034] running on GCE node
I20260810 07:18:21.086923 8424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:21.087556 8424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:21.088722 8424 hybrid_clock.cc:648] HybridClock initialized: now 1786346301088693 us; error 38 us; skew 500 ppm
Aug 10 07:18:21 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:21.091202 8424 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:18:21.092029 8424 webserver.cc:460] Webserver started at http://127.12.117.131:40571/ using document root <none> and password file <none>
I20260810 07:18:21.092533 8424 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:21.092623 8424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:21.096186 8424 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260810 07:18:21.098133 8450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:21.099354 8424 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:21.099460 8424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "f3179b78a91c4625b538722eacc9e726"
format_stamp: "Formatted at 2026-08-10 07:18:00 on dist-test-slave-gjrz"
server_key: "003da725465ac7adc9203979cc6f4e90"
server_key_iv: "b75ceb4b24ad7c0737a25039d9633237"
server_key_version: "encryptionkey@0"
I20260810 07:18:21.099963 8424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:21.112993 8424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:21.115646 8424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:21.115800 8424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:21.116314 8424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:21.117057 8424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:21.117103 8424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:21.117152 8424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:21.117178 8424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:21.125504 8424 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41117
I20260810 07:18:21.125547 8667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41117 every 8 connection(s)
I20260810 07:18:21.126390 8424 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:21.131923 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 8424
Aug 10 07:18:21 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:21.139355 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.128048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:52945 (local address 127.12.117.190:43387)
0810 07:18:21.128174 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.128178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.128949 (+ 771us) server_negotiation.cc:407] Connection header received
0810 07:18:21.130183 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.130187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.130246 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.130315 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:21.131428 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.131964 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.133310 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.133532 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.133619 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.135861 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:21.135864 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:21.135866 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:21.135888 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:21.137246 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.137813 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.137817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.137819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.137915 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.138273 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.138276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.138277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.138466 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:21.138556 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.139038 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.139167 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:18:21.140069 8670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.128364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:52945)
0810 07:18:21.128788 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.128828 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:21.129984 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:21.130500 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:21.130507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:21.130824 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:21.131252 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.131286 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.132109 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.132111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.133191 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.133202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.133663 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.133667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.133831 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.134312 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:21.134329 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:21.135709 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:21.137358 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:21.137362 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:21.137373 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:21.137709 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:21.138037 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:21.138040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:21.138043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:21.138176 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:21.138574 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:21.138581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:21.138802 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:18:21.138980 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 07:18:21.139300 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":99,"threads_started":1}
I20260810 07:18:21.140528 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.132525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.12.117.190:43387)
0810 07:18:21.132706 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.132710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.132953 (+ 243us) server_negotiation.cc:407] Connection header received
0810 07:18:21.133050 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.133053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.133096 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.133170 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:21.133911 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.134419 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.135246 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.135550 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.135638 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.137008 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:21.137012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:21.137014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:21.137049 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:21.138510 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.138885 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.138889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.138891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.138970 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.139280 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.139284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.139286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.139507 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:21.139697 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.139966 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.140078 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
I20260810 07:18:21.141144 8668 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43387
I20260810 07:18:21.141361 8668 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:21.141856 8668 heartbeater.cc:511] Master 127.12.117.190:43387 requested a full tablet report, sending...
I20260810 07:18:21.142836 7829 ts_manager.cc:194] Registered new tserver with Master: f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
I20260810 07:18:21.143400 7829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:52945
I20260810 07:18:21.153800 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:21.157713 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.154367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.12.117.190:43387)
0810 07:18:21.154508 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.154512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.154655 (+ 143us) server_negotiation.cc:407] Connection header received
0810 07:18:21.154819 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.154821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.154905 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.155002 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:21.155230 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.155757 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.156636 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.156859 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.156946 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.157247 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:21.157362 (+ 115us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 07:18:21.157597 (+ 235us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42292: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:18:21.157768 8671 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42292: token expired
W20260810 07:18:21.158054 7703 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.117.190:43387)
I20260810 07:18:21.164597 8671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.158364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.12.117.190:43387)
0810 07:18:21.158474 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.158478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.158490 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:18:21.158588 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.158590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.158628 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.158688 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:21.159298 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.159763 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.160508 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.160715 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.160796 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.161962 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:21.161966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:21.161968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:21.161993 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:21.163075 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.163522 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.163525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.163527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.163604 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:21.163858 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:21.163861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:21.163863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:21.164023 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:21.164137 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.164358 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.164468 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:18:21.166723 7829 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:21.169812 7829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:21.177032 8042 tablet_service.cc:1467] Processing CreateTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 (DEFAULT_TABLE table=test-table [id=51520a63d9224bbc8ab74203df99006c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:21.177664 8042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785d01016b984a58bff9da9dcda2c9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:21.180321 7970 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap starting.
I20260810 07:18:21.180970 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.175825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38913 (local address 127.0.0.1:53732)
0810 07:18:21.175931 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.175945 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:21.176026 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:21.176831 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:21.176834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:21.176853 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:21.176941 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.176948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.178276 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.178279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.179477 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.179489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.180582 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.180586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.180681 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.180729 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 07:18:21.180777 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 07:18:21.180855 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 07:18:21.181113 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.176118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41117 (local address 127.0.0.1:60662)
0810 07:18:21.176404 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.176416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:21.176503 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:21.177340 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:21.177342 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:21.177360 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:21.177442 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.177448 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.178646 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.178649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.179715 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.179725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.180717 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.180720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.180817 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.180842 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:18:21.180913 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:18:21.180979 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":87,"threads_started":1}
I20260810 07:18:21.181166 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.176426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.12.117.131:41117)
0810 07:18:21.176863 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.176875 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.176896 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:18:21.176992 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.176997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.177186 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.177333 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:21.177556 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.178546 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.179849 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.180604 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.180701 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.180822 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.180941 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.181041 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":153,"threads_started":1}
I20260810 07:18:21.181206 7970 tablet_bootstrap.cc:654] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:21.182556 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.176013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.12.117.130:38913)
0810 07:18:21.176334 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.176346 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.176371 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:18:21.176450 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.176456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.176673 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.176832 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:21.177043 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.178150 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.179610 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.180426 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.181149 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.182262 (+ 1113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.182314 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.182400 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":108,"threads_started":1}
I20260810 07:18:21.182772 7970 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: No bootstrap required, opened a new log
I20260810 07:18:21.182859 7970 ts_tablet_manager.cc:1397] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:21.183231 7970 raft_consensus.cc:348] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.183363 7970 raft_consensus.cc:374] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:21.183409 7970 raft_consensus.cc:729] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390b49248b7942bfaf2fa795cfbf3f9a, State: Initialized, Role: FOLLOWER
I20260810 07:18:21.183527 7970 consensus_queue.cc:260] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.183773 7970 ts_tablet_manager.cc:1428] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 07:18:21.184206 8542 tablet_service.cc:1467] Processing CreateTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 (DEFAULT_TABLE table=test-table [id=51520a63d9224bbc8ab74203df99006c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:21.185283 8542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785d01016b984a58bff9da9dcda2c9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:21.185310 8295 tablet_service.cc:1467] Processing CreateTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 (DEFAULT_TABLE table=test-table [id=51520a63d9224bbc8ab74203df99006c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:21.186299 8295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785d01016b984a58bff9da9dcda2c9c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:21.193096 8692 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Bootstrap starting.
I20260810 07:18:21.193781 8693 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Bootstrap starting.
I20260810 07:18:21.195317 8692 tablet_bootstrap.cc:654] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:21.196393 8693 tablet_bootstrap.cc:654] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:21.196645 8692 log.cc:824] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:21.197190 8693 log.cc:824] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:21.198592 8692 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: No bootstrap required, opened a new log
I20260810 07:18:21.198784 8692 ts_tablet_manager.cc:1397] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:18:21.198988 8693 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: No bootstrap required, opened a new log
I20260810 07:18:21.199172 8693 ts_tablet_manager.cc:1397] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:18:21.202521 8692 raft_consensus.cc:348] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.202805 8692 raft_consensus.cc:374] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:21.202867 8692 raft_consensus.cc:729] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3179b78a91c4625b538722eacc9e726, State: Initialized, Role: FOLLOWER
I20260810 07:18:21.202759 8693 raft_consensus.cc:348] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.202999 8693 raft_consensus.cc:374] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:21.203063 8693 raft_consensus.cc:729] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9d9bada3fc04730a6825221f76d17f0, State: Initialized, Role: FOLLOWER
I20260810 07:18:21.203387 8692 consensus_queue.cc:260] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.203503 8693 consensus_queue.cc:260] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.203990 8668 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
I20260810 07:18:21.204082 8420 heartbeater.cc:503] Master 127.12.117.190:43387 was elected leader, sending a full tablet report...
I20260810 07:18:21.204383 8693 ts_tablet_manager.cc:1428] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 07:18:21.204383 8692 ts_tablet_manager.cc:1428] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:18:21.204479 8703 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:21.204597 8703 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
W20260810 07:18:21.204779 8421 tablet.cc:2369] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:21.207727 8703 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117), 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637)
I20260810 07:18:21.210544 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.207149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41117 (local address 127.12.117.130:49305)
0810 07:18:21.207246 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.207287 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:21.207363 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:21.207786 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:21.207789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:21.207806 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:21.207910 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.207916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.208807 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.208811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.209560 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.209569 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.210165 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.210169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.210271 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.210293 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:18:21.210357 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:18:21.210426 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260810 07:18:21.210543 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.207385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:49305 (local address 127.12.117.131:41117)
0810 07:18:21.207492 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.207497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.207513 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:18:21.207555 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.207557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.207594 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.207673 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:21.208009 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.208706 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.209677 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.210065 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.210164 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.210213 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.210378 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.210442 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:18:21.211068 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.207630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:36847 (local address 127.12.117.129:35637)
0810 07:18:21.207740 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.207744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.208031 (+ 287us) server_negotiation.cc:407] Connection header received
0810 07:18:21.208119 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.208122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.208177 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.208251 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:21.208501 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.209133 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.210102 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.210531 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.210634 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.210766 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.210890 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.210962 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:21.211051 8582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785d01016b984a58bff9da9dcda2c9c2" candidate_uuid: "f9d9bada3fc04730a6825221f76d17f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3179b78a91c4625b538722eacc9e726" is_pre_election: true
I20260810 07:18:21.211297 8582 raft_consensus.cc:2454] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d9bada3fc04730a6825221f76d17f0 in term 0.
I20260810 07:18:21.211411 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.207530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:35637 (local address 127.12.117.130:36847)
0810 07:18:21.207934 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.207945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:21.208040 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:21.208269 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:21.208271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:21.208288 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:21.208393 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.208397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.209236 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.209239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.209988 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:21.209997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.210650 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.210654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:21.210747 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.210770 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:18:21.211195 (+ 425us) client_negotiation.cc:240] Negotiation successful
0810 07:18:21.211286 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":103,"threads_started":1}
I20260810 07:18:21.211737 8209 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0; no voters:
I20260810 07:18:21.211944 8703 raft_consensus.cc:2781] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:21.212018 8703 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:21.211939 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785d01016b984a58bff9da9dcda2c9c2" candidate_uuid: "f9d9bada3fc04730a6825221f76d17f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" is_pre_election: true
I20260810 07:18:21.212049 8703 raft_consensus.cc:3037] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:21.212183 8075 raft_consensus.cc:2454] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9d9bada3fc04730a6825221f76d17f0 in term 0.
I20260810 07:18:21.213155 8703 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.213438 8703 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [CANDIDATE]: Term 1 election: Requested vote from peers f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117), 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637)
I20260810 07:18:21.213780 8582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785d01016b984a58bff9da9dcda2c9c2" candidate_uuid: "f9d9bada3fc04730a6825221f76d17f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3179b78a91c4625b538722eacc9e726"
I20260810 07:18:21.213807 8075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785d01016b984a58bff9da9dcda2c9c2" candidate_uuid: "f9d9bada3fc04730a6825221f76d17f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
I20260810 07:18:21.213901 8582 raft_consensus.cc:3037] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:21.213903 8075 raft_consensus.cc:3037] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:21.215049 8582 raft_consensus.cc:2454] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d9bada3fc04730a6825221f76d17f0 in term 1.
I20260810 07:18:21.215050 8075 raft_consensus.cc:2454] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9d9bada3fc04730a6825221f76d17f0 in term 1.
I20260810 07:18:21.215349 8209 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0; no voters:
I20260810 07:18:21.215530 8703 raft_consensus.cc:2781] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:21.215622 8703 raft_consensus.cc:686] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 1 LEADER]: Becoming Leader. State: Replica: f9d9bada3fc04730a6825221f76d17f0, State: Running, Role: LEADER
I20260810 07:18:21.215885 8703 consensus_queue.cc:237] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:21.218648 7829 catalog_manager.cc:5495] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130). New cstate: current_term: 1 leader_uuid: "f9d9bada3fc04730a6825221f76d17f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } health_report { overall_health: UNKNOWN } } }
I20260810 07:18:21.227999 8683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.225139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53748 (local address 127.12.117.130:38913)
0810 07:18:21.225230 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:21.225232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:21.225271 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:18:21.225386 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:21.225388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:21.225432 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:21.225509 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:21.225659 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.226104 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.226861 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:21.227068 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:21.227137 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:21.227389 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:21.227709 (+ 320us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:21.227792 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:21.227852 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 07:18:21.227894 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:18:21.232796 8582 raft_consensus.cc:1261] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Refusing update from remote peer f9d9bada3fc04730a6825221f76d17f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:21.232795 8075 raft_consensus.cc:1261] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Refusing update from remote peer f9d9bada3fc04730a6825221f76d17f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:21.233407 8703 consensus_queue.cc:1035] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:21.233587 8709 consensus_queue.cc:1035] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:21.246403 7829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:21.246556 7829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:21.248730 7829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Sending DeleteTablet for 3 replicas of tablet 785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:21.249310 8295 tablet_service.cc:1514] Processing DeleteTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:21 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:53732
I20260810 07:18:21.249446 8542 tablet_service.cc:1514] Processing DeleteTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:21 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:60662
I20260810 07:18:21.249874 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 7893
I20260810 07:18:21.249922 8042 tablet_service.cc:1514] Processing DeleteTablet for tablet 785d01016b984a58bff9da9dcda2c9c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:21 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:59258
I20260810 07:18:21.250016 8729 tablet_replica.cc:330] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: stopping tablet replica
I20260810 07:18:21.250092 8185 tablet_replica.cc:330] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: stopping tablet replica
I20260810 07:18:21.250193 8185 raft_consensus.cc:2229] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:21.250257 8729 raft_consensus.cc:2229] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:21.250555 8729 raft_consensus.cc:2258] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:21.255493 8729 ts_tablet_manager.cc:1905] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:21.257792 8729 ts_tablet_manager.cc:1918] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:21.257862 8729 log.cc:1196] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:21.258116 8729 ts_tablet_manager.cc:1939] T 785d01016b984a58bff9da9dcda2c9c2 P f9d9bada3fc04730a6825221f76d17f0: Deleting consensus metadata
I20260810 07:18:21.258585 8730 tablet_replica.cc:330] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: stopping tablet replica
I20260810 07:18:21.258903 8730 raft_consensus.cc:2229] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:21.259117 7804 catalog_manager.cc:4841] TS f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): tablet 785d01016b984a58bff9da9dcda2c9c2 (table test-table [id=51520a63d9224bbc8ab74203df99006c]) successfully deleted
I20260810 07:18:21.259320 8730 raft_consensus.cc:2258] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:21.264279 8730 ts_tablet_manager.cc:1905] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:21.266192 8730 ts_tablet_manager.cc:1918] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:21.266273 8730 log.cc:1196] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:21.266543 8730 ts_tablet_manager.cc:1939] T 785d01016b984a58bff9da9dcda2c9c2 P f3179b78a91c4625b538722eacc9e726: Deleting consensus metadata
I20260810 07:18:21.267472 7801 catalog_manager.cc:4841] TS f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): tablet 785d01016b984a58bff9da9dcda2c9c2 (table test-table [id=51520a63d9224bbc8ab74203df99006c]) successfully deleted
W20260810 07:18:21.268297 7801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:35637 (error 108)
I20260810 07:18:21.268879 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 8173
I20260810 07:18:21.269299 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:21.268986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:35637 (local address 127.0.0.1:59272)
0810 07:18:21.269154 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:21.269223 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:35637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260810 07:18:21.269447 7801 catalog_manager.cc:4584] TS 390b49248b7942bfaf2fa795cfbf3f9a (127.12.117.129:35637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 785d01016b984a58bff9da9dcda2c9c2: Network error: Client connection negotiation failed: client connection to 127.12.117.129:35637: connect: Connection refused (error 111)
I20260810 07:18:21.282243 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 8424
I20260810 07:18:21.294863 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 7768
I20260810 07:18:41.308319 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--webserver_interface=127.12.117.190
--webserver_port=39121
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:41.421293 8732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.421527 8732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.421579 8732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.425457 8732 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 07:18:41.425503 8732 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 07:18:41.425524 8732 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 07:18:41.425540 8732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:41.425565 8732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.425596 8732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.425621 8732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:41.425648 8732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:41.430692 8732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=39121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:41.431704 8732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.432644 8732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:41.441226 8732 server_base.cc:1034] running on GCE node
W20260810 07:18:41.441195 8744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.441188 8743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.441171 8746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.441946 8732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.442965 8732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.444120 8732 hybrid_clock.cc:648] HybridClock initialized: now 1786346321444098 us; error 34 us; skew 500 ppm
Aug 10 07:18:41 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:41.446779 8732 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:18:41.447629 8732 webserver.cc:460] Webserver started at http://127.12.117.190:39121/ using document root <none> and password file <none>
I20260810 07:18:41.448148 8732 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.448215 8732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.451764 8732 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:18:41.453747 8759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.454919 8732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:41.455049 8732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2d00d0303e0549c185c23d607c83d7bd"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "c509d5a10bb2f1579ee31c55dfd68dcc"
server_key_iv: "82c47d7c0ba2f1c6f1cbf691764e0fc1"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.455598 8732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.522609 8732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.525440 8732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.525619 8732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.532454 8732 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:43387
I20260810 07:18:41.532722 8868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:43387 every 8 connection(s)
I20260810 07:18:41.533497 8732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:41.536199 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 8732
I20260810 07:18:41.536599 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:35637
--local_ip_for_outbound_sockets=127.12.117.129
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=39999
--webserver_interface=127.12.117.129
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:41.537735 8871 sys_catalog.cc:263] Verifying existing consensus state
I20260810 07:18:41.539371 8871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap starting.
I20260810 07:18:41.555871 8871 log.cc:824] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.566561 8871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:41.566867 8871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: Bootstrap complete.
I20260810 07:18:41.570555 8871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:41.570904 8871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.571394 8871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:41.571511 8871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:41.571570 8871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:41.571663 8871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 2 FOLLOWER]: Advancing to term 3
I20260810 07:18:41.573339 8871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:41.573635 8871 leader_election.cc:302] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d00d0303e0549c185c23d607c83d7bd; no voters:
I20260810 07:18:41.573891 8871 leader_election.cc:288] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 07:18:41.573977 8878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 3 FOLLOWER]: Leader election won for term 3
I20260810 07:18:41.574160 8878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [term 3 LEADER]: Becoming Leader. State: Replica: 2d00d0303e0549c185c23d607c83d7bd, State: Running, Role: LEADER
I20260810 07:18:41.574436 8878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } }
I20260810 07:18:41.575023 8871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:41.575349 8878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d00d0303e0549c185c23d607c83d7bd. Latest consensus state: current_term: 3 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:18:41.575541 8878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:41.576861 8882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2d00d0303e0549c185c23d607c83d7bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d00d0303e0549c185c23d607c83d7bd" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43387 } } }
I20260810 07:18:41.576963 8882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:41.579404 8902 catalog_manager.cc:1259] Loaded cluster ID: ae91d31e91ce4e629dc9ca875b0b8ac7
I20260810 07:18:41.579448 8902 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: loading cluster ID for follower catalog manager: success
I20260810 07:18:41.581136 8902 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d00d0303e0549c185c23d607c83d7bd: acquiring CA information for follower catalog manager: success
F20260810 07:18:41.582037 8902 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcaf2fb09ad google::LogMessage::Fail() at ??:0
@ 0x7fcaf2fb388a google::LogMessage::SendToLog() at ??:0
@ 0x7fcaf2fb04de google::LogMessage::Flush() at ??:0
@ 0x7fcaf2fb1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcaf5c7418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fcafd915989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fcafd8f2023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fcafd8f254b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fcafd8eb652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fcafd8ea9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fcafd931541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fcaf51f3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcaf68376db start_thread at ??:0
@ 0x7fcaf1aae61f clone at ??:0
W20260810 07:18:41.660346 8874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.660591 8874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.660650 8874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.664595 8874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.664649 8874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.664721 8874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:18:41.669852 8874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:35637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=39999
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:41.670789 8874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.671770 8874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:41.682103 8938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.682096 8939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.682320 8874 server_base.cc:1034] running on GCE node
W20260810 07:18:41.682466 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.682909 8874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.683668 8874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.684870 8874 hybrid_clock.cc:648] HybridClock initialized: now 1786346321684831 us; error 43 us; skew 500 ppm
Aug 10 07:18:41 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:41.688002 8874 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:18:41.689074 8874 webserver.cc:460] Webserver started at http://127.12.117.129:39999/ using document root <none> and password file <none>
I20260810 07:18:41.689761 8874 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.689850 8874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.694761 8874 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:41.697332 8953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.698753 8874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:18:41.698873 8874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "390b49248b7942bfaf2fa795cfbf3f9a"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "f4dfff004326b387c0d035c5df3e4328"
server_key_iv: "7808250b3d502ff2507f1fcbd978ab6b"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.699566 8874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.742543 8874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.746279 8874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.746487 8874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.747165 8874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:41.748495 8968 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 07:18:41.752550 8874 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 07:18:41.752626 8874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260810 07:18:41.752702 8874 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 07:18:41.754920 8874 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 07:18:41.754980 8874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 07:18:41.755288 8968 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap starting.
I20260810 07:18:41.766070 8874 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:35637
I20260810 07:18:41.766309 9179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:35637 every 8 connection(s)
I20260810 07:18:41.767251 8874 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:41.773893 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 8874
I20260810 07:18:41.774394 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:38913
--local_ip_for_outbound_sockets=127.12.117.130
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=33987
--webserver_interface=127.12.117.130
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:41.774508 8968 log.cc:824] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:41.782091 8968 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 07:18:41.782471 8968 tablet_bootstrap.cc:492] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Bootstrap complete.
I20260810 07:18:41.783800 8968 ts_tablet_manager.cc:1397] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent bootstrapping tablet: real 0.029s user 0.018s sys 0.010s
I20260810 07:18:41.787024 8968 raft_consensus.cc:348] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:41.787597 8968 raft_consensus.cc:729] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390b49248b7942bfaf2fa795cfbf3f9a, State: Initialized, Role: FOLLOWER
I20260810 07:18:41.788156 8968 consensus_queue.cc:260] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:41.789276 8968 ts_tablet_manager.cc:1428] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
W20260810 07:18:41.802738 9186 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:18:41.769573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:55001)
0810 07:18:41.770155 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.770214 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:41.771279 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:41.801956 (+ 30677us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":158,"threads_started":1}
W20260810 07:18:41.803200 9180 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:43387 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:43387: Client connection negotiation failed: client connection to 127.12.117.190:43387: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 07:18:41.803892 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.803657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:40983)
0810 07:18:41.803776 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.803837 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:18:41.804517 9186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.804326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:54529)
0810 07:18:41.804423 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.804468 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:18:41.804669 9180 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:18:41.892212 9191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:41.892438 9191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:41.892505 9191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:41.896428 9191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:41.896556 9191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:41.896687 9191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:18:41.901980 9191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=33987
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:41.902884 9191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:41.903833 9191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:41.913019 9191 server_base.cc:1034] running on GCE node
W20260810 07:18:41.912985 9199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.912986 9205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:41.912993 9201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:41.913676 9191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:41.914268 9191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:41.915412 9191 hybrid_clock.cc:648] HybridClock initialized: now 1786346321915387 us; error 37 us; skew 500 ppm
Aug 10 07:18:41 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:41.917999 9191 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:18:41.918820 9191 webserver.cc:460] Webserver started at http://127.12.117.130:33987/ using document root <none> and password file <none>
I20260810 07:18:41.919363 9191 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:41.919435 9191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:41.923023 9191 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:18:41.925029 9224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.926235 9191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:18:41.926345 9191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "f9d9bada3fc04730a6825221f76d17f0"
format_stamp: "Formatted at 2026-08-10 07:17:59 on dist-test-slave-gjrz"
server_key: "ccb9cca669663ecee773bbc5e315a54a"
server_key_iv: "5215e4794acf87fa1b0a08cafc6fee22"
server_key_version: "encryptionkey@0"
I20260810 07:18:41.926878 9191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:41.936309 9191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:41.939101 9191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:41.939258 9191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:41.939829 9191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:41.940584 9191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:41.940631 9191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.940680 9191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:41.940709 9191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:41.949024 9191 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:38913
I20260810 07:18:41.949049 9449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:38913 every 8 connection(s)
I20260810 07:18:41.949955 9191 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:41.953513 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.951904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:51947)
0810 07:18:41.952327 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.952430 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
W20260810 07:18:41.953914 9450 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:43387 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:43387: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
I20260810 07:18:41.953956 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 9191
I20260810 07:18:41.954437 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:41117
--local_ip_for_outbound_sockets=127.12.117.131
--tserver_master_addrs=127.12.117.190:43387
--webserver_port=40571
--webserver_interface=127.12.117.131
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:18:41.954507 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.954299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:49227)
0810 07:18:41.954407 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.954452 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260810 07:18:41.955155 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:41.954980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:50681)
0810 07:18:41.955061 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:41.955109 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:18:41.955345 9450 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 07:18:42.063655 9453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:42.063886 9453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:42.063925 9453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:42.067844 9453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:42.067894 9453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:42.067950 9453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:18:42.072969 9453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:41117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=40571
--tserver_master_addrs=127.12.117.190:43387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:42.073936 9453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:42.074868 9453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:18:42.084306 9453 server_base.cc:1034] running on GCE node
W20260810 07:18:42.084227 9463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:42.084220 9464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:42.084220 9466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:42.084898 9453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:42.085459 9453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:42.086601 9453 hybrid_clock.cc:648] HybridClock initialized: now 1786346322086577 us; error 36 us; skew 500 ppm
Aug 10 07:18:42 dist-test-slave-gjrz krb5kdc[6805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346322, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:42.089389 9453 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:18:42.090265 9453 webserver.cc:460] Webserver started at http://127.12.117.131:40571/ using document root <none> and password file <none>
I20260810 07:18:42.090795 9453 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:42.090862 9453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:42.094473 9453 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:42.096644 9478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:42.097771 9453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:42.097873 9453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "f3179b78a91c4625b538722eacc9e726"
format_stamp: "Formatted at 2026-08-10 07:18:00 on dist-test-slave-gjrz"
server_key: "003da725465ac7adc9203979cc6f4e90"
server_key_iv: "b75ceb4b24ad7c0737a25039d9633237"
server_key_version: "encryptionkey@0"
I20260810 07:18:42.098400 9453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:42.117136 9453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:42.120118 9453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:42.120287 9453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:42.120841 9453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:42.121623 9453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:42.121672 9453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:42.121724 9453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:42.121750 9453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:42.130051 9453 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:41117
I20260810 07:18:42.130120 9689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:41117 every 8 connection(s)
I20260810 07:18:42.130999 9453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:42.134574 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:42.132959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:60325)
0810 07:18:42.133389 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:42.133495 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":108,"threads_started":1}
W20260810 07:18:42.134929 9690 heartbeater.cc:644] Failed to heartbeat to 127.12.117.190:43387 (0 consecutive failures): Network error: Failed to ping master at 127.12.117.190:43387: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
I20260810 07:18:42.135444 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:42.135242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:45193)
0810 07:18:42.135351 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:42.135394 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:18:42.135907 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:42.135755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:36745)
0810 07:18:42.135829 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:42.135865 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:18:42.136060 9690 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 07:18:42.141129 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 9453
I20260810 07:18:42.805562 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:42.805129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:42415)
0810 07:18:42.805399 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:42.805468 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
I20260810 07:18:42.956197 9697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:42.955797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:46143)
0810 07:18:42.956041 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:42.956094 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":63,"threads_started":1}
I20260810 07:18:43.092536 9699 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:43.092759 9699 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:43.093809 9699 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
Aug 10 07:18:43 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.131@KRBTEST.COM
Aug 10 07:18:43 dist-test-slave-gjrz krb5kdc[6805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.130@KRBTEST.COM
I20260810 07:18:43.106938 9702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.094007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:41117 (local address 127.12.117.129:39457)
0810 07:18:43.094321 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.094336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.094438 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.096026 (+ 1588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.096033 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.096555 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:43.097022 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.097036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.098294 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.098296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.099361 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.099372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.100050 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.100053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.100279 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.100876 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:43.100892 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:43.102379 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:43.104679 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.104681 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.104682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.104954 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.105271 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105273 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.105274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.105419 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.106021 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:43.106024 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:43.106247 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.106402 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.106726 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":251,"spinlock_wait_cycles":33024,"thread_start_us":68,"threads_started":1}
I20260810 07:18:43.106948 9703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.094191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:59229 (local address 127.12.117.130:38913)
0810 07:18:43.094598 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.094620 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.094658 (+ 38us) server_negotiation.cc:407] Connection header received
0810 07:18:43.095331 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.095337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.095676 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.095957 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.097185 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.098154 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.099492 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.099958 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.100104 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.102898 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.102902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.102913 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.102939 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.104467 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105043 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.105049 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.105055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.105172 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105518 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.105521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.105523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.105912 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.106022 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.106532 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.106798 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":69,"threads_started":1}
I20260810 07:18:43.106987 9705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.094248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:39457 (local address 127.12.117.131:41117)
0810 07:18:43.094630 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:43.094640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:43.094673 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:18:43.095320 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:43.095327 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:43.095677 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:43.095968 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:43.097189 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.098173 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.099488 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.099938 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.100083 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.102606 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:43.102613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:43.102628 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:43.102663 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:43.104574 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105059 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.105064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.105070 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.105180 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105512 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:43.105516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:43.105518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:43.105910 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:43.106036 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:43.106543 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 07:18:43.106803 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
I20260810 07:18:43.106980 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.094007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38913 (local address 127.12.117.129:59229)
0810 07:18:43.094265 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.094280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:43.094388 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:43.096036 (+ 1648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:43.096039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:43.096555 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:43.097022 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.097036 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.098307 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.098310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.099351 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:43.099361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:43.100074 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:43.100078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:43.100265 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:43.100853 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:43.100877 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:43.102729 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:43.104562 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.104568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.104582 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.104937 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.105265 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:43.105269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:43.105297 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:43.105426 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:43.106014 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:43.106020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:43.106248 (+ 228us) client_negotiation.cc:769] Sending connection context
0810 07:18:43.106403 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 07:18:43.106804 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":224,"spinlock_wait_cycles":26880}
W20260810 07:18:43.109648 8962 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
W20260810 07:18:43.109895 8960 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:43.109992 8960 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a; no voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0
I20260810 07:18:43.110245 9699 raft_consensus.cc:2726] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:43.136863 9706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.136518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:57505)
0810 07:18:43.136726 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.136776 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260810 07:18:43.806644 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.806235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:55655)
0810 07:18:43.806473 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.806537 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":73,"threads_started":1}
I20260810 07:18:43.957217 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:43.956831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:52361)
0810 07:18:43.957078 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:43.957131 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":79,"threads_started":1}
I20260810 07:18:44.137907 9713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.137513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:51789)
0810 07:18:44.137749 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.137805 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
I20260810 07:18:44.807734 9715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.807340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:57067)
0810 07:18:44.807579 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.807638 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":68,"threads_started":1}
I20260810 07:18:44.958346 9716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:44.957944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:38993)
0810 07:18:44.958201 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:44.958255 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260810 07:18:45.085215 9717 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:45.085340 9717 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:45.085745 9717 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
W20260810 07:18:45.086499 8960 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
W20260810 07:18:45.086613 8962 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:45.086702 8962 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a; no voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0
I20260810 07:18:45.086846 9717 raft_consensus.cc:2726] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:45.138909 9718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:45.138542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:59831)
0810 07:18:45.138778 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:45.138826 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260810 07:18:45.808789 9719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:45.808401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:39727)
0810 07:18:45.808640 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:45.808693 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260810 07:18:45.959408 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:45.958999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:52053)
0810 07:18:45.959241 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:45.959317 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260810 07:18:46.139921 9723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.139543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:59791)
0810 07:18:46.139779 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.139834 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260810 07:18:46.810017 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.809579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:37219)
0810 07:18:46.809863 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.809916 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":89,"threads_started":1}
I20260810 07:18:46.960494 9725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:46.960088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:48353)
0810 07:18:46.960322 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:46.960376 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":86,"threads_started":1}
I20260810 07:18:47.141067 9727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.140607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:48689)
0810 07:18:47.140908 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.140965 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
I20260810 07:18:47.811121 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.810719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:48733)
0810 07:18:47.810974 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.811027 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260810 07:18:47.961596 9730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:47.961190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:53909)
0810 07:18:47.961439 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:47.961503 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":86,"threads_started":1}
I20260810 07:18:48.140504 9731 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:48.140648 9731 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:48.141045 9731 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
W20260810 07:18:48.141808 8962 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
W20260810 07:18:48.142007 8960 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:48.142069 8960 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a; no voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0
I20260810 07:18:48.142081 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:48.141696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:56345)
0810 07:18:48.141920 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:48.141990 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":67,"threads_started":1}
I20260810 07:18:48.142230 9731 raft_consensus.cc:2726] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:48.812196 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:48.811778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:57503)
0810 07:18:48.812036 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:48.812101 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260810 07:18:48.962659 9734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:48.962236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:35781)
0810 07:18:48.962507 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:48.962560 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":70,"threads_started":1}
I20260810 07:18:49.143136 9735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:49.142699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:42787)
0810 07:18:49.142999 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:49.143051 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":82,"threads_started":1}
I20260810 07:18:49.813196 9736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:49.812816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:35057)
0810 07:18:49.813049 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:49.813101 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
I20260810 07:18:49.963694 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:49.963319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:55755)
0810 07:18:49.963552 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:49.963604 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260810 07:18:50.144244 9740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:50.143819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:57957)
0810 07:18:50.144098 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:50.144154 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260810 07:18:50.814277 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:50.813863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:50243)
0810 07:18:50.814126 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:50.814180 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260810 07:18:50.964753 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:50.964373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:41141)
0810 07:18:50.964608 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:50.964661 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":84,"threads_started":1}
I20260810 07:18:51.145270 9743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:51.144878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:59133)
0810 07:18:51.145131 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:51.145184 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
I20260810 07:18:51.815464 9744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:51.815072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:33727)
0810 07:18:51.815316 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:51.815369 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":74,"threads_started":1}
I20260810 07:18:51.965852 9746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:51.965459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:56591)
0810 07:18:51.965700 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:51.965757 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":94,"threads_started":1}
I20260810 07:18:52.146298 9748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:52.145923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:50415)
0810 07:18:52.146153 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:52.146203 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260810 07:18:52.786152 9749 raft_consensus.cc:482] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:52.786273 9749 raft_consensus.cc:504] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9d9bada3fc04730a6825221f76d17f0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38913 } } peers { permanent_uuid: "f3179b78a91c4625b538722eacc9e726" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 41117 } } peers { permanent_uuid: "390b49248b7942bfaf2fa795cfbf3f9a" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 35637 } }
I20260810 07:18:52.786661 9749 leader_election.cc:288] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913), f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117)
W20260810 07:18:52.787433 8962 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f9d9bada3fc04730a6825221f76d17f0 (127.12.117.130:38913): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
W20260810 07:18:52.787566 8960 leader_election.cc:341] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f3179b78a91c4625b538722eacc9e726 (127.12.117.131:41117): Not found: Tablet not found: 785d01016b984a58bff9da9dcda2c9c2
I20260810 07:18:52.787631 8960 leader_election.cc:302] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 390b49248b7942bfaf2fa795cfbf3f9a; no voters: f3179b78a91c4625b538722eacc9e726, f9d9bada3fc04730a6825221f76d17f0
I20260810 07:18:52.787753 9749 raft_consensus.cc:2726] T 785d01016b984a58bff9da9dcda2c9c2 P 390b49248b7942bfaf2fa795cfbf3f9a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 07:18:52.816442 9752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:52.816078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:49831)
0810 07:18:52.816296 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:52.816357 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
I20260810 07:18:52.966887 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:52.966502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:37005)
0810 07:18:52.966751 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:52.966801 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260810 07:18:53.147507 9755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:53.147032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:35093)
0810 07:18:53.147313 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:53.147390 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260810 07:18:53.817507 9757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:53.817126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:37553)
0810 07:18:53.817361 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:53.817412 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260810 07:18:53.967979 9759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:53.967605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:41823)
0810 07:18:53.967838 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:53.967889 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260810 07:18:54.148586 9760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:54.148188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:59151)
0810 07:18:54.148437 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:54.148488 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":91,"threads_started":1}
I20260810 07:18:54.818557 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:54.818175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:44709)
0810 07:18:54.818411 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:54.818463 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260810 07:18:54.969043 9763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:54.968648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:37557)
0810 07:18:54.968872 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:54.968926 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260810 07:18:55.149739 9764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:55.149324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:59667)
0810 07:18:55.149581 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:55.149638 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260810 07:18:55.819700 9767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:55.819296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:55683)
0810 07:18:55.819533 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:55.819596 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":78,"threads_started":1}
I20260810 07:18:55.970039 9769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:55.969661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:35849)
0810 07:18:55.969893 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:55.969945 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260810 07:18:56.150837 9770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:56.150430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:53575)
0810 07:18:56.150685 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:56.150741 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260810 07:18:56.820865 9772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:56.820494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.129:58135)
0810 07:18:56.820719 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:56.820770 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":80,"threads_started":1}
I20260810 07:18:56.971217 9773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:56.970815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.130:49365)
0810 07:18:56.971078 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:56.971129 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
I20260810 07:18:57.143745 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 8874
I20260810 07:18:57.152036 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:57.151665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43387 (local address 127.12.117.131:53249)
0810 07:18:57.151885 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:57.151936 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.190:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
I20260810 07:18:57.156855 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 9191
I20260810 07:18:57.167701 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 9453
2026-08-10T07:18:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59122 ms)
[----------] 30 tests from SecurityITest (239355 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:18:57 dist-test-slave-gjrz krb5kdc[9782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:18:57 dist-test-slave-gjrz krb5kdc[9782](info): set up 2 sockets
Aug 10 07:18:57 dist-test-slave-gjrz krb5kdc[9782](info): commencing operation
krb5kdc: starting...
W20260810 07:18:58.346163 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:18:58 dist-test-slave-gjrz krb5kdc[9782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:18:58.459681 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43395
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:43395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:58.568265 9809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:58.568492 9809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:58.568529 9809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:58.572376 9809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:58.572422 9809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:58.572441 9809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:58.572471 9809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:58.572496 9809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:58.577539 9809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:43395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:43395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:58.578425 9809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:58.579352 9809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:58.587534 9824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:58.587543 9819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:58.587543 9820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:58.587651 9809 server_base.cc:1034] running on GCE node
I20260810 07:18:58.588281 9809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:58.589250 9809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:58.590396 9809 hybrid_clock.cc:648] HybridClock initialized: now 1786346338590372 us; error 37 us; skew 500 ppm
Aug 10 07:18:58 dist-test-slave-gjrz krb5kdc[9782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346338, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:58.592917 9809 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:18:58.593708 9809 webserver.cc:460] Webserver started at http://127.12.117.190:43345/ using document root <none> and password file <none>
I20260810 07:18:58.594208 9809 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:58.594269 9809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:58.594458 9809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:58.596385 9809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "3a1c9cc54b62499abd2c1e54f257c39f"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "aff3e47bbb1e28d201f5deb68fc43a2d"
server_key_iv: "1ad8212e62187e5876fd7aa37c4db038"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.596901 9809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "3a1c9cc54b62499abd2c1e54f257c39f"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "aff3e47bbb1e28d201f5deb68fc43a2d"
server_key_iv: "1ad8212e62187e5876fd7aa37c4db038"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.600519 9809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 07:18:58.602906 9842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:58.604110 9809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:58.604218 9809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "3a1c9cc54b62499abd2c1e54f257c39f"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "aff3e47bbb1e28d201f5deb68fc43a2d"
server_key_iv: "1ad8212e62187e5876fd7aa37c4db038"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.604322 9809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:58.638329 9809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:58.641165 9809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:58.641325 9809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:58.648106 9809 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:43395
I20260810 07:18:58.648195 9936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:43395 every 8 connection(s)
I20260810 07:18:58.649013 9809 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:18:58.651742 9938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:58.655696 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 9809
I20260810 07:18:58.655831 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:18:58.656121 12758 external_mini_cluster.cc:1428] Setting key 85d9ce51913402f82bdff49ca5ee1007
I20260810 07:18:58.657385 9938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: Bootstrap starting.
I20260810 07:18:58.659688 9938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:58.660426 9938 log.cc:824] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:58.662495 9938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: No bootstrap required, opened a new log
Aug 10 07:18:58 dist-test-slave-gjrz krb5kdc[9782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:58.666373 9938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } }
I20260810 07:18:58.666671 9938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:58.666729 9938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1c9cc54b62499abd2c1e54f257c39f, State: Initialized, Role: FOLLOWER
I20260810 07:18:58.667201 9938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } }
I20260810 07:18:58.667383 9938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:18:58.667448 9938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:18:58.667549 9938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:58.668332 9938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } }
I20260810 07:18:58.668674 9938 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1c9cc54b62499abd2c1e54f257c39f; no voters:
I20260810 07:18:58.668991 9938 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:18:58.669123 9947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:58.669445 9947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [term 1 LEADER]: Becoming Leader. State: Replica: 3a1c9cc54b62499abd2c1e54f257c39f, State: Running, Role: LEADER
I20260810 07:18:58.669837 9947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } }
I20260810 07:18:58.670187 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:58.657355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.12.117.190:43395)
0810 07:18:58.657746 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:58.657765 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:58.657800 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:18:58.658401 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:58.658417 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:58.658739 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:58.659010 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:58.659926 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.660917 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.662152 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.662829 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.662993 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:58.665343 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:58.665349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:58.665366 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:58.665398 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:58.667203 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:58.667716 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:58.667722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:58.667727 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:58.667839 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:58.668116 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:58.668119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:58.668121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:58.668545 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:58.668719 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:58.669054 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 07:18:58.669349 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260810 07:18:58.670272 9938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:18:58.671541 9949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } } }
I20260810 07:18:58.671659 9949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [sys.catalog]: This master's current role is: LEADER
I20260810 07:18:58.671871 9949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1c9cc54b62499abd2c1e54f257c39f. Latest consensus state: current_term: 1 leader_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1c9cc54b62499abd2c1e54f257c39f" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 43395 } } }
I20260810 07:18:58.671945 9949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f [sys.catalog]: This master's current role is: LEADER
W20260810 07:18:58.674942 9977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:18:58.675004 9977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:18:58.675081 9966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:18:58.675836 9966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:18:58.680945 9966 catalog_manager.cc:1347] Generated new cluster ID: 18dfaff20f4741709a6e6c93f1f67437
I20260810 07:18:58.681002 9966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:18:58.706041 9966 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:18:58.707211 9966 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:18:58.728987 9966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: Generated new TSK 0
I20260810 07:18:58.729599 9966 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:18:58.761492 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:58.867743 9988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:58.867964 9988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:58.868021 9988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:58.872020 9988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:58.872082 9988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:58.872141 9988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:18:58.877211 9988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:58.878062 9988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:58.878934 9988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:58.888113 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:58.888156 9996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:58.888223 9988 server_base.cc:1034] running on GCE node
W20260810 07:18:58.888175 9998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:58.888602 9988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:58.889181 9988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:58.890337 9988 hybrid_clock.cc:648] HybridClock initialized: now 1786346338890323 us; error 27 us; skew 500 ppm
Aug 10 07:18:58 dist-test-slave-gjrz krb5kdc[9782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346338, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:58.893043 9988 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:18:58.893908 9988 webserver.cc:460] Webserver started at http://127.12.117.129:37409/ using document root <none> and password file <none>
I20260810 07:18:58.894385 9988 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:58.894441 9988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:58.894613 9988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:58.896510 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "c12d1e003a574dd59e257d8291598665"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "6075dc9620cb669fa0450df69e93c613"
server_key_iv: "c1a9cb133ff5f199d4abc7d6037dfe62"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.896999 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "c12d1e003a574dd59e257d8291598665"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "6075dc9620cb669fa0450df69e93c613"
server_key_iv: "c1a9cb133ff5f199d4abc7d6037dfe62"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.900373 9988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:18:58.902568 10007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:58.903749 9988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:18:58.903851 9988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "c12d1e003a574dd59e257d8291598665"
format_stamp: "Formatted at 2026-08-10 07:18:58 on dist-test-slave-gjrz"
server_key: "6075dc9620cb669fa0450df69e93c613"
server_key_iv: "c1a9cb133ff5f199d4abc7d6037dfe62"
server_key_version: "encryptionkey@0"
I20260810 07:18:58.903928 9988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:58.918790 9988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:58.921398 9988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:58.921545 9988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:58.922027 9988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:58.922752 9988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:58.922797 9988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:58.922832 9988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:58.922852 9988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:58.931074 9988 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:44931
I20260810 07:18:58.931108 10214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:44931 every 8 connection(s)
I20260810 07:18:58.931885 9988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:18:58.937145 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 9988
I20260810 07:18:58.937243 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:18:58.937565 12758 external_mini_cluster.cc:1428] Setting key 4a5ff6bc0ae14cb58a6f27dcb4b9ec39
Aug 10 07:18:58 dist-test-slave-gjrz krb5kdc[9782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346338, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:58.944721 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:58.933524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:40057 (local address 127.12.117.190:43395)
0810 07:18:58.933642 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:58.933646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:58.934395 (+ 749us) server_negotiation.cc:407] Connection header received
0810 07:18:58.935586 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:58.935589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:58.935634 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:58.935703 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:58.936734 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.937275 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.938656 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.938932 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.939024 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:58.941440 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:58.941443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:58.941445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:58.941476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:58.942821 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:58.943316 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:58.943319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:58.943320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:58.943396 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:58.943701 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:58.943705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:58.943706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:58.943877 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:58.943960 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:58.944428 (+ 468us) server_negotiation.cc:299] Negotiation successful
0810 07:18:58.944564 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:58.945427 10220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:58.933839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43395 (local address 127.12.117.129:40057)
0810 07:18:58.934257 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:58.934297 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:58.935409 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:58.935840 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:58.935846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:58.936187 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:58.936620 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:58.936633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.937397 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.937401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:58.938554 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:58.938565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:58.939082 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:58.939087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:58.939237 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:58.939763 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:58.939778 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:58.941286 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:58.942932 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:58.942938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:58.942951 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:58.943197 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:58.943480 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:58.943484 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:58.943486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:58.943614 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:58.943983 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:58.943989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:58.944208 (+ 219us) client_negotiation.cc:769] Sending connection context
0810 07:18:58.944369 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 07:18:58.944664 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":112,"threads_started":1}
I20260810 07:18:58.946640 10215 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43395
I20260810 07:18:58.946842 10215 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:18:58.947324 10215 heartbeater.cc:511] Master 127.12.117.190:43395 requested a full tablet report, sending...
I20260810 07:18:58.948727 9872 ts_manager.cc:194] Registered new tserver with Master: c12d1e003a574dd59e257d8291598665 (127.12.117.129:44931)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:18:58.949970 9872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:40057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:18:58.963727 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:59.071882 10223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:59.072118 10223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:59.072160 10223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:59.076124 10223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:59.076181 10223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:59.076236 10223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:18:59.081321 10223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:59.082283 10223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:59.083213 10223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:59.092300 10234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:59.092300 10232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:59.092394 10229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:59.092630 10223 server_base.cc:1034] running on GCE node
I20260810 07:18:59.093012 10223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:59.093587 10223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:59.094732 10223 hybrid_clock.cc:648] HybridClock initialized: now 1786346339094702 us; error 45 us; skew 500 ppm
Aug 10 07:18:59 dist-test-slave-gjrz krb5kdc[9782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:59.097303 10223 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:18:59.098116 10223 webserver.cc:460] Webserver started at http://127.12.117.130:36169/ using document root <none> and password file <none>
I20260810 07:18:59.098631 10223 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:59.098696 10223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:59.098875 10223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:59.100842 10223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "130ee49f082d43bca6dfd9060d7b9751"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "26a71b0079bab72dc415edb6eed52f0d"
server_key_iv: "a005c27763a5619081329784e7723c14"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.101366 10223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "130ee49f082d43bca6dfd9060d7b9751"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "26a71b0079bab72dc415edb6eed52f0d"
server_key_iv: "a005c27763a5619081329784e7723c14"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.104866 10223 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 07:18:59.107029 10250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.108266 10223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 07:18:59.108385 10223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "130ee49f082d43bca6dfd9060d7b9751"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "26a71b0079bab72dc415edb6eed52f0d"
server_key_iv: "a005c27763a5619081329784e7723c14"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.108487 10223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:59.152405 10223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:59.155110 10223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:59.155257 10223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:59.155790 10223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:59.156538 10223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:59.156585 10223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.156623 10223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:59.156644 10223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.164875 10223 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:40207
I20260810 07:18:59.164904 10437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:40207 every 8 connection(s)
I20260810 07:18:59.165684 10223 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:18:59.169083 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 10223
I20260810 07:18:59.169184 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:18:59.169435 12758 external_mini_cluster.cc:1428] Setting key 0c8d312a53909d07ee3fc79cc4ff0527
Aug 10 07:18:59 dist-test-slave-gjrz krb5kdc[9782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:18:59.179844 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.167382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:56839 (local address 127.12.117.190:43395)
0810 07:18:59.167506 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.167509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.168289 (+ 780us) server_negotiation.cc:407] Connection header received
0810 07:18:59.169723 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.169726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.169781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.169850 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:59.171287 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.172033 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.173492 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.173732 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.173811 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.176525 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:59.176528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:59.176531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:59.176555 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:59.177854 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.178337 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.178340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.178341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.178419 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.178741 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.178745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.178746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.178925 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:59.179020 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.179554 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.179680 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:18:59.180644 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.167725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43395 (local address 127.12.117.130:56839)
0810 07:18:59.168145 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.168185 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.169464 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.170044 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.170051 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.170550 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:59.171108 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.171122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.172186 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.172190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.173372 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.173385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.173873 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.173877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.174075 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.174616 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:59.174632 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:59.176358 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:59.177967 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:59.177973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:59.177987 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:59.178236 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:59.178511 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:59.178514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:59.178516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:59.178651 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:59.179025 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:59.179032 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:59.179298 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.179492 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.179832 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":110,"threads_started":1}
I20260810 07:18:59.181676 10438 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:18:59.181937 10438 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:59.182435 10438 heartbeater.cc:511] Master 127.12.117.190:43395 requested a full tablet report, sending...
I20260810 07:18:59.183467 9872 ts_manager.cc:194] Registered new tserver with Master: 130ee49f082d43bca6dfd9060d7b9751 (127.12.117.130:40207)
I20260810 07:18:59.184201 9872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:56839
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:18:59.195870 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:18:59.301970 10447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:59.302191 10447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:59.302235 10447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:59.306249 10447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:59.306300 10447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:59.306370 10447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:18:59.311510 10447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:43395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:59.312386 10447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:18:59.313277 10447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:18:59.322091 10460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:18:59.322227 10467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:59.322278 10447 server_base.cc:1034] running on GCE node
W20260810 07:18:59.322183 10458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:18:59.322700 10447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:18:59.323335 10447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:18:59.324489 10447 hybrid_clock.cc:648] HybridClock initialized: now 1786346339324477 us; error 29 us; skew 500 ppm
Aug 10 07:18:59 dist-test-slave-gjrz krb5kdc[9782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:18:59.327013 10447 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:18:59.327826 10447 webserver.cc:460] Webserver started at http://127.12.117.131:43411/ using document root <none> and password file <none>
I20260810 07:18:59.328338 10447 fs_manager.cc:359] Metadata directory not provided
I20260810 07:18:59.328401 10447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:18:59.328579 10447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:18:59.330425 10447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "58654a245b2649bf881360e6a1d189d2"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "34a132a501bc38f507374e3f9f68d3d0"
server_key_iv: "2f359651594974b118d67e60681198e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.330935 10447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "58654a245b2649bf881360e6a1d189d2"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "34a132a501bc38f507374e3f9f68d3d0"
server_key_iv: "2f359651594974b118d67e60681198e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.334314 10447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:18:59.336488 10481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.337566 10447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:18:59.337669 10447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "58654a245b2649bf881360e6a1d189d2"
format_stamp: "Formatted at 2026-08-10 07:18:59 on dist-test-slave-gjrz"
server_key: "34a132a501bc38f507374e3f9f68d3d0"
server_key_iv: "2f359651594974b118d67e60681198e2"
server_key_version: "encryptionkey@0"
I20260810 07:18:59.337757 10447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:18:59.356112 10447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:18:59.358882 10447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:18:59.359045 10447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:18:59.359581 10447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:18:59.360319 10447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:18:59.360364 10447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.360414 10447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:18:59.360443 10447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:18:59.368522 10447 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:45003
I20260810 07:18:59.368546 10684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:45003 every 8 connection(s)
I20260810 07:18:59.369294 10447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:18:59.370988 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 10447
I20260810 07:18:59.371120 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:18:59.371392 12758 external_mini_cluster.cc:1428] Setting key 1e8b188f2b9612df2d1d6415b542f9fa
Aug 10 07:18:59 dist-test-slave-gjrz krb5kdc[9782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346339, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:18:59.381448 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.370900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:35905 (local address 127.12.117.190:43395)
0810 07:18:59.371068 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.371073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.371958 (+ 885us) server_negotiation.cc:407] Connection header received
0810 07:18:59.373588 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.373591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.373646 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.373713 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:59.374723 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.375237 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.376325 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.376533 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.376601 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.378743 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:59.378746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:59.378748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:59.378771 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:59.379909 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.380327 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.380329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.380330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.380395 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.380637 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.380639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.380641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.380792 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:59.380859 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.381243 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.381334 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 07:18:59.382179 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.371318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:43395 (local address 127.12.117.131:35905)
0810 07:18:59.371777 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.371870 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.373384 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.373859 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.373866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.374183 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:18:59.374614 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.374627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.375358 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.375361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.376233 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.376241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.376631 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.376633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.376759 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.377233 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:18:59.377249 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:18:59.378593 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:18:59.380010 (+ 1417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:59.380015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:59.380025 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:59.380255 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:59.380455 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:18:59.380457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:18:59.380458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:18:59.380569 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:18:59.380858 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:18:59.380862 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:18:59.381063 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.381204 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.381422 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":127,"threads_started":1}
I20260810 07:18:59.383149 10685 heartbeater.cc:347] Connected to a master server at 127.12.117.190:43395
I20260810 07:18:59.383386 10685 heartbeater.cc:464] Registering TS with master...
I20260810 07:18:59.383828 10685 heartbeater.cc:511] Master 127.12.117.190:43395 requested a full tablet report, sending...
I20260810 07:18:59.384873 9872 ts_manager.cc:194] Registered new tserver with Master: 58654a245b2649bf881360e6a1d189d2 (127.12.117.131:45003)
I20260810 07:18:59.385430 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:18:59.385622 9872 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:35905
I20260810 07:18:59.393425 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.387534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.12.117.190:43395)
0810 07:18:59.387654 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.387659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.387811 (+ 152us) server_negotiation.cc:407] Connection header received
0810 07:18:59.387910 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.387912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.387945 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.388003 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:18:59.388563 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.389026 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.389757 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.389949 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.390012 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.391115 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:18:59.391118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:18:59.391120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:18:59.391133 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:18:59.392207 (+ 1074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.392544 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.392547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.392549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.392628 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:18:59.392806 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:18:59.392808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:18:59.392809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:18:59.392984 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:18:59.393076 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.393216 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.393297 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:18:59.395877 9872 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:18:59.398651 9872 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:18:59.412359 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.407291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54028 (local address 127.12.117.129:44931)
0810 07:18:59.407645 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.407658 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.407681 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:18:59.407756 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.407761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.407962 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.408119 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:59.408460 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409580 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.410884 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.411691 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.411817 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.411943 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.412128 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.412222 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":88,"threads_started":1}
I20260810 07:18:59.412417 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.406578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:44931 (local address 127.0.0.1:54028)
0810 07:18:59.407397 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.407409 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.407482 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.408264 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.408267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.408282 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:59.408357 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.408363 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409747 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.409750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.410768 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.410778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.411816 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.411820 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.411922 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.411964 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.412065 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.412185 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"mutex_wait_us":28,"thread_start_us":103,"threads_started":1}
I20260810 07:18:59.412806 10702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.406213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:40207 (local address 127.0.0.1:38332)
0810 07:18:59.406518 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.406567 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.406699 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.408038 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.408044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.408068 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:59.408193 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.408207 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409609 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.409613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.410976 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.410985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.412026 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.412029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.412525 (+ 496us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.412548 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.412626 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.412691 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":501,"thread_start_us":83,"threads_started":1}
I20260810 07:18:59.412915 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.406868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.12.117.130:40207)
0810 07:18:59.407514 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.407526 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.407549 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:18:59.407625 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.407631 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.407803 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.408006 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:59.408312 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409471 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.411114 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.411861 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.411972 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.412088 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.412675 (+ 587us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.412765 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":120,"threads_started":1}
I20260810 07:18:59.412426 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.406314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:45003 (local address 127.0.0.1:45234)
0810 07:18:59.407136 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.407149 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.407235 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.408135 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.408138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.408156 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:59.408256 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.408263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409433 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.409437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.410974 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.410985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.412026 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.412029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.412161 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.412183 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.412242 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.412307 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":126,"thread_start_us":72,"threads_started":1}
I20260810 07:18:59.413676 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.406954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45234 (local address 127.12.117.131:45003)
0810 07:18:59.407726 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.407757 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.407775 (+ 18us) server_negotiation.cc:407] Connection header received
0810 07:18:59.407831 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.407836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.407971 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.408398 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:59.408529 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.409317 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.411094 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.411894 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.413080 (+ 1186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.413424 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.413471 (+ 47us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.413548 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":145,"threads_started":1}
I20260810 07:18:59.415320 10094 tablet_service.cc:1467] Processing CreateTablet for tablet d00577aad2ff40e195d99e08a13b696b (DEFAULT_TABLE table=test-table [id=b4d7b72411fc4edb9d10ef7780e3aa13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:59.416216 10094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00577aad2ff40e195d99e08a13b696b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:59.415803 10334 tablet_service.cc:1467] Processing CreateTablet for tablet d00577aad2ff40e195d99e08a13b696b (DEFAULT_TABLE table=test-table [id=b4d7b72411fc4edb9d10ef7780e3aa13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:59.416057 10558 tablet_service.cc:1467] Processing CreateTablet for tablet d00577aad2ff40e195d99e08a13b696b (DEFAULT_TABLE table=test-table [id=b4d7b72411fc4edb9d10ef7780e3aa13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:18:59.416828 10334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00577aad2ff40e195d99e08a13b696b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:59.416915 10558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d00577aad2ff40e195d99e08a13b696b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:18:59.421998 10717 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Bootstrap starting.
I20260810 07:18:59.423718 10721 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Bootstrap starting.
I20260810 07:18:59.424418 10717 tablet_bootstrap.cc:654] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:59.424664 10725 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Bootstrap starting.
I20260810 07:18:59.425683 10717 log.cc:824] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:59.425925 10721 tablet_bootstrap.cc:654] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:59.426384 10725 tablet_bootstrap.cc:654] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Neither blocks nor log segments found. Creating new log.
I20260810 07:18:59.427013 10721 log.cc:824] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:59.427426 10725 log.cc:824] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Log is configured to *not* fsync() on all Append() calls
I20260810 07:18:59.427738 10717 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: No bootstrap required, opened a new log
I20260810 07:18:59.427929 10717 ts_tablet_manager.cc:1397] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:18:59.428781 10721 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: No bootstrap required, opened a new log
I20260810 07:18:59.428943 10721 ts_tablet_manager.cc:1397] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:18:59.430794 10717 raft_consensus.cc:348] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.430997 10717 raft_consensus.cc:374] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:59.431061 10717 raft_consensus.cc:729] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12d1e003a574dd59e257d8291598665, State: Initialized, Role: FOLLOWER
I20260810 07:18:59.431480 10717 consensus_queue.cc:260] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.431730 10721 raft_consensus.cc:348] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.431977 10721 raft_consensus.cc:374] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:59.432032 10721 raft_consensus.cc:729] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58654a245b2649bf881360e6a1d189d2, State: Initialized, Role: FOLLOWER
I20260810 07:18:59.432149 10215 heartbeater.cc:503] Master 127.12.117.190:43395 was elected leader, sending a full tablet report...
I20260810 07:18:59.432422 10717 ts_tablet_manager.cc:1428] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:18:59.432495 10721 consensus_queue.cc:260] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
W20260810 07:18:59.432849 10219 tablet.cc:2369] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:59.432914 10725 tablet_bootstrap.cc:492] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: No bootstrap required, opened a new log
I20260810 07:18:59.433077 10725 ts_tablet_manager.cc:1397] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260810 07:18:59.433681 10685 heartbeater.cc:503] Master 127.12.117.190:43395 was elected leader, sending a full tablet report...
I20260810 07:18:59.434002 10721 ts_tablet_manager.cc:1428] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:18:59.437094 10725 raft_consensus.cc:348] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.437296 10725 raft_consensus.cc:374] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:18:59.437341 10725 raft_consensus.cc:729] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130ee49f082d43bca6dfd9060d7b9751, State: Initialized, Role: FOLLOWER
I20260810 07:18:59.437711 10725 consensus_queue.cc:260] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.438273 10438 heartbeater.cc:503] Master 127.12.117.190:43395 was elected leader, sending a full tablet report...
I20260810 07:18:59.438426 10725 ts_tablet_manager.cc:1428] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
W20260810 07:18:59.620198 10686 tablet.cc:2369] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:18:59.666687 10439 tablet.cc:2369] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:18:59.679973 10730 raft_consensus.cc:482] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:18:59.680172 10730 raft_consensus.cc:504] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.681298 10730 leader_election.cc:288] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 130ee49f082d43bca6dfd9060d7b9751 (127.12.117.130:40207), 58654a245b2649bf881360e6a1d189d2 (127.12.117.131:45003)
I20260810 07:18:59.685184 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.681652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:43605 (local address 127.12.117.131:45003)
0810 07:18:59.681837 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.681842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.682005 (+ 163us) server_negotiation.cc:407] Connection header received
0810 07:18:59.682130 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.682132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.682178 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.682252 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:59.682512 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.683015 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684208 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.684587 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684690 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.684746 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.684989 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.685063 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260810 07:18:59.685190 10711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.681597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:37561 (local address 127.12.117.130:40207)
0810 07:18:59.681722 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.681727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.681947 (+ 220us) server_negotiation.cc:407] Connection header received
0810 07:18:59.682071 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.682075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.682138 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.682198 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:18:59.682542 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.683024 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684223 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.684593 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684696 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.684746 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.684992 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.685063 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 07:18:59.685398 10736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.681542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:45003 (local address 127.12.117.129:43605)
0810 07:18:59.681903 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.681917 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.682024 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.682279 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.682282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.682298 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:59.682411 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.682416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.683126 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.683129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.684093 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.684102 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.684696 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.684871 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.684897 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.684963 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.685063 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":135,"thread_start_us":95,"threads_started":1}
I20260810 07:18:59.685676 10739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.681502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:40207 (local address 127.12.117.129:37561)
0810 07:18:59.681835 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.681847 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:18:59.681967 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:18:59.682228 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:18:59.682232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:18:59.682248 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:18:59.682427 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.682433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.683128 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.683131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.684107 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:18:59.684116 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.684699 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.684702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:18:59.684873 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.684899 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:18:59.684962 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:18:59.685074 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":152,"thread_start_us":96,"threads_started":1}
I20260810 07:18:59.686026 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00577aad2ff40e195d99e08a13b696b" candidate_uuid: "c12d1e003a574dd59e257d8291598665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58654a245b2649bf881360e6a1d189d2" is_pre_election: true
I20260810 07:18:59.686223 10365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00577aad2ff40e195d99e08a13b696b" candidate_uuid: "c12d1e003a574dd59e257d8291598665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ee49f082d43bca6dfd9060d7b9751" is_pre_election: true
I20260810 07:18:59.686329 10592 raft_consensus.cc:2454] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c12d1e003a574dd59e257d8291598665 in term 0.
I20260810 07:18:59.686498 10365 raft_consensus.cc:2454] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c12d1e003a574dd59e257d8291598665 in term 0.
I20260810 07:18:59.686800 10010 leader_election.cc:302] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58654a245b2649bf881360e6a1d189d2, c12d1e003a574dd59e257d8291598665; no voters:
I20260810 07:18:59.687036 10730 raft_consensus.cc:2781] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:18:59.687119 10730 raft_consensus.cc:482] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:18:59.687168 10730 raft_consensus.cc:3037] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:59.688165 10730 raft_consensus.cc:504] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.688454 10730 leader_election.cc:288] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [CANDIDATE]: Term 1 election: Requested vote from peers 130ee49f082d43bca6dfd9060d7b9751 (127.12.117.130:40207), 58654a245b2649bf881360e6a1d189d2 (127.12.117.131:45003)
I20260810 07:18:59.688767 10365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00577aad2ff40e195d99e08a13b696b" candidate_uuid: "c12d1e003a574dd59e257d8291598665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ee49f082d43bca6dfd9060d7b9751"
I20260810 07:18:59.688822 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d00577aad2ff40e195d99e08a13b696b" candidate_uuid: "c12d1e003a574dd59e257d8291598665" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58654a245b2649bf881360e6a1d189d2"
I20260810 07:18:59.688876 10365 raft_consensus.cc:3037] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:59.688933 10592 raft_consensus.cc:3037] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:18:59.689905 10592 raft_consensus.cc:2454] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c12d1e003a574dd59e257d8291598665 in term 1.
I20260810 07:18:59.689961 10365 raft_consensus.cc:2454] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c12d1e003a574dd59e257d8291598665 in term 1.
I20260810 07:18:59.690215 10010 leader_election.cc:302] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58654a245b2649bf881360e6a1d189d2, c12d1e003a574dd59e257d8291598665; no voters:
I20260810 07:18:59.690382 10730 raft_consensus.cc:2781] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:18:59.690572 10730 raft_consensus.cc:686] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 1 LEADER]: Becoming Leader. State: Replica: c12d1e003a574dd59e257d8291598665, State: Running, Role: LEADER
I20260810 07:18:59.690913 10730 consensus_queue.cc:237] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } }
I20260810 07:18:59.694115 9868 catalog_manager.cc:5495] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 reported cstate change: term changed from 0 to 1, leader changed from <none> to c12d1e003a574dd59e257d8291598665 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "c12d1e003a574dd59e257d8291598665" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c12d1e003a574dd59e257d8291598665" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 44931 } health_report { overall_health: HEALTHY } } }
I20260810 07:18:59.770152 10716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.766713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.12.117.129:44931)
0810 07:18:59.766839 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:18:59.766844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:18:59.766858 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:18:59.766992 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:18:59.766995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:18:59.767054 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:18:59.767154 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:18:59.767436 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.767930 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.768778 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:18:59.768990 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:18:59.769077 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:18:59.769374 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:18:59.769780 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:18:59.769888 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:18:59.769956 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 07:18:59.770014 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:18:59.775741 10365 raft_consensus.cc:1261] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 1 FOLLOWER]: Refusing update from remote peer c12d1e003a574dd59e257d8291598665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:59.775741 10592 raft_consensus.cc:1261] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 1 FOLLOWER]: Refusing update from remote peer c12d1e003a574dd59e257d8291598665: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:18:59.776418 10743 consensus_queue.cc:1035] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58654a245b2649bf881360e6a1d189d2" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 45003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:59.776595 10730 consensus_queue.cc:1035] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [LEADER]: Connected to new peer: Peer: permanent_uuid: "130ee49f082d43bca6dfd9060d7b9751" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:18:59.783697 10746 mvcc.cc:205] Tried to move back new op lower bound from 7316874607714729984 to 7316874607376994304. Current Snapshot: MvccSnapshot[applied={T|T < 7316874607714729984}]
I20260810 07:18:59.783941 10753 mvcc.cc:205] Tried to move back new op lower bound from 7316874607714729984 to 7316874607376994304. Current Snapshot: MvccSnapshot[applied={T|T < 7316874607714729984}]
I20260810 07:18:59.784801 10751 mvcc.cc:205] Tried to move back new op lower bound from 7316874607714729984 to 7316874607376994304. Current Snapshot: MvccSnapshot[applied={T|T < 7316874607714729984}]
I20260810 07:18:59.792315 9870 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:59.792498 9870 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:18:59.794821 9870 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a1c9cc54b62499abd2c1e54f257c39f: Sending DeleteTablet for 3 replicas of tablet d00577aad2ff40e195d99e08a13b696b
I20260810 07:18:59.795524 10334 tablet_service.cc:1514] Processing DeleteTablet for tablet d00577aad2ff40e195d99e08a13b696b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:59 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:38332
I20260810 07:18:59.795524 10558 tablet_service.cc:1514] Processing DeleteTablet for tablet d00577aad2ff40e195d99e08a13b696b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:59 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:45234
I20260810 07:18:59.795694 10094 tablet_service.cc:1514] Processing DeleteTablet for tablet d00577aad2ff40e195d99e08a13b696b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:18:59 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:54028
I20260810 07:18:59.795998 10764 tablet_replica.cc:330] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: stopping tablet replica
I20260810 07:18:59.796124 10766 tablet_replica.cc:330] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665: stopping tablet replica
I20260810 07:18:59.796236 10765 tablet_replica.cc:330] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: stopping tablet replica
I20260810 07:18:59.796407 10766 raft_consensus.cc:2229] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:18:59.796676 10765 raft_consensus.cc:2229] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:59.796783 10766 raft_consensus.cc:2258] T d00577aad2ff40e195d99e08a13b696b P c12d1e003a574dd59e257d8291598665 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:59.796829 10764 raft_consensus.cc:2229] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:18:59.796907 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 9988
I20260810 07:18:59.797199 10764 raft_consensus.cc:2258] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:59.797585 10765 raft_consensus.cc:2258] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:18:59.804986 10764 ts_tablet_manager.cc:1905] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:59.808393 10764 ts_tablet_manager.cc:1918] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:59.808523 10764 log.cc:1196] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/d00577aad2ff40e195d99e08a13b696b
I20260810 07:18:59.808938 10764 ts_tablet_manager.cc:1939] T d00577aad2ff40e195d99e08a13b696b P 130ee49f082d43bca6dfd9060d7b9751: Deleting consensus metadata
I20260810 07:18:59.810524 9843 catalog_manager.cc:4841] TS 130ee49f082d43bca6dfd9060d7b9751 (127.12.117.130:40207): tablet d00577aad2ff40e195d99e08a13b696b (table test-table [id=b4d7b72411fc4edb9d10ef7780e3aa13]) successfully deleted
I20260810 07:18:59.819868 10765 ts_tablet_manager.cc:1905] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:18:59.822008 10765 ts_tablet_manager.cc:1918] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:18:59.822103 10765 log.cc:1196] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/d00577aad2ff40e195d99e08a13b696b
I20260810 07:18:59.822372 10765 ts_tablet_manager.cc:1939] T d00577aad2ff40e195d99e08a13b696b P 58654a245b2649bf881360e6a1d189d2: Deleting consensus metadata
I20260810 07:18:59.823467 9845 catalog_manager.cc:4841] TS 58654a245b2649bf881360e6a1d189d2 (127.12.117.131:45003): tablet d00577aad2ff40e195d99e08a13b696b (table test-table [id=b4d7b72411fc4edb9d10ef7780e3aa13]) successfully deleted
W20260810 07:18:59.824810 9849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:44931 (error 108)
I20260810 07:18:59.825065 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 10223
I20260810 07:18:59.826030 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:18:59.825748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:44931 (local address 127.0.0.1:41732)
0810 07:18:59.825899 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 07:18:59.825954 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:44931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 07:18:59.826187 9849 catalog_manager.cc:4584] TS c12d1e003a574dd59e257d8291598665 (127.12.117.129:44931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d00577aad2ff40e195d99e08a13b696b: Network error: Client connection negotiation failed: client connection to 127.12.117.129:44931: connect: Connection refused (error 111)
I20260810 07:18:59.835898 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 10447
I20260810 07:18:59.849751 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 9809
2026-08-10T07:18:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2682 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T07:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:18:59Z Disabled control of system clock
I20260810 07:18:59.882282 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38721
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:38721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:18:59.990123 10776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:18:59.990348 10776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:18:59.990382 10776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:18:59.994155 10776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:18:59.994197 10776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:18:59.994211 10776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:18:59.994225 10776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:18:59.994238 10776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:18:59.998740 10776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:38721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:18:59.999642 10776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:00.000873 10776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:00.009006 10791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.008992 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.009063 10776 server_base.cc:1034] running on GCE node
W20260810 07:19:00.008992 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.010032 10776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:00.011029 10776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:00.012172 10776 hybrid_clock.cc:648] HybridClock initialized: now 1786346340012149 us; error 35 us; skew 500 ppm
I20260810 07:19:00.013937 10776 webserver.cc:460] Webserver started at http://127.12.117.190:44131/ using document root <none> and password file <none>
I20260810 07:19:00.014477 10776 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:00.014539 10776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:00.014741 10776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:00.016726 10776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "aa8dee124c0b4c3fadc349439e4ba5da"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f867f487c12f23efc2c0b5dd231dc18f"
server_key_iv: "39e6e28ce4b10f6189d988212ca757da"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.017269 10776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "aa8dee124c0b4c3fadc349439e4ba5da"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f867f487c12f23efc2c0b5dd231dc18f"
server_key_iv: "39e6e28ce4b10f6189d988212ca757da"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.020624 10776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:00.022861 10803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.023998 10776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:00.024142 10776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "aa8dee124c0b4c3fadc349439e4ba5da"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f867f487c12f23efc2c0b5dd231dc18f"
server_key_iv: "39e6e28ce4b10f6189d988212ca757da"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.024237 10776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:00.042953 10776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:00.043606 10776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:00.043768 10776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:00.050240 10776 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:38721
I20260810 07:19:00.050259 10904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:38721 every 8 connection(s)
I20260810 07:19:00.051132 10776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:00.054152 10905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:00.057739 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 10776
I20260810 07:19:00.057853 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:00.058128 12758 external_mini_cluster.cc:1428] Setting key d24ddeadeb0509c5e8ea9ff70937eba5
I20260810 07:19:00.059595 10905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: Bootstrap starting.
I20260810 07:19:00.062182 10905 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:00.063026 10905 log.cc:824] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:00.064884 10905 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: No bootstrap required, opened a new log
I20260810 07:19:00.068008 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.059475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36652 (local address 127.12.117.190:38721)
0810 07:19:00.060115 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.060135 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.060176 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:00.060987 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.061010 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.061478 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.061827 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.062290 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.063665 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.064534 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.065220 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.065369 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.066085 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.066092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.066107 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.066569 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.066649 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.066676 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.066769 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.066838 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.067090 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
I20260810 07:19:00.068884 10905 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } }
I20260810 07:19:00.069162 10905 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:00.069221 10905 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8dee124c0b4c3fadc349439e4ba5da, State: Initialized, Role: FOLLOWER
I20260810 07:19:00.069654 10905 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } }
I20260810 07:19:00.069799 10905 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:00.069864 10905 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:00.069952 10905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:00.070720 10905 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } }
I20260810 07:19:00.071008 10905 leader_election.cc:302] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa8dee124c0b4c3fadc349439e4ba5da; no voters:
I20260810 07:19:00.071383 10905 leader_election.cc:288] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:00.071455 10915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:00.071667 10915 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [term 1 LEADER]: Becoming Leader. State: Replica: aa8dee124c0b4c3fadc349439e4ba5da, State: Running, Role: LEADER
I20260810 07:19:00.071969 10915 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } }
I20260810 07:19:00.072535 10905 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:00.073478 10917 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa8dee124c0b4c3fadc349439e4ba5da. Latest consensus state: current_term: 1 leader_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } } }
I20260810 07:19:00.073468 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8dee124c0b4c3fadc349439e4ba5da" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38721 } } }
I20260810 07:19:00.073616 10917 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:00.073616 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:00.073978 10927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:00.076789 10927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:00.082068 10927 catalog_manager.cc:1347] Generated new cluster ID: c86f2fd4eca7484dbbd742a681ad91dc
I20260810 07:19:00.082139 10927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:00.095675 10927 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:00.096505 10927 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:00.102007 10927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: Generated new TSK 0
I20260810 07:19:00.102566 10927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:00.108672 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--builtin_ntp_servers=127.12.117.148:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:19:00.217857 10941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:00.218091 10941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:00.218128 10941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:00.222083 10941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:00.222138 10941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:00.222190 10941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:19:00.226747 10941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:00.227699 10941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:00.228976 10941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:00.238219 10950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.238301 10941 server_base.cc:1034] running on GCE node
W20260810 07:19:00.238204 10949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.238207 10952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.238907 10941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:00.239527 10941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:00.240674 10941 hybrid_clock.cc:648] HybridClock initialized: now 1786346340240651 us; error 39 us; skew 500 ppm
I20260810 07:19:00.242419 10941 webserver.cc:460] Webserver started at http://127.12.117.129:43795/ using document root <none> and password file <none>
I20260810 07:19:00.242961 10941 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:00.243027 10941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:00.243222 10941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:00.245180 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "b2f71f82a7ce4c2080c971e7944c0a2b"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f0a91a461868e649d4150d83b315c7e9"
server_key_iv: "1c0b53bfc5c44a2997f21d531eae7ce6"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.245700 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "b2f71f82a7ce4c2080c971e7944c0a2b"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f0a91a461868e649d4150d83b315c7e9"
server_key_iv: "1c0b53bfc5c44a2997f21d531eae7ce6"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.249197 10941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:00.251446 10961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.252556 10941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:00.252671 10941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "b2f71f82a7ce4c2080c971e7944c0a2b"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f0a91a461868e649d4150d83b315c7e9"
server_key_iv: "1c0b53bfc5c44a2997f21d531eae7ce6"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.252761 10941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:00.283571 10941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:00.284190 10941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:00.284351 10941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:00.284890 10941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:00.285776 10941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:00.285823 10941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.285874 10941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:00.285903 10941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.294096 10941 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:37963
I20260810 07:19:00.294121 11153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:37963 every 8 connection(s)
I20260810 07:19:00.294898 10941 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:00.295228 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 10941
I20260810 07:19:00.295395 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:00.295753 12758 external_mini_cluster.cc:1428] Setting key da83306c3242cc63fe3f27a9993fedc3
I20260810 07:19:00.297808 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--builtin_ntp_servers=127.12.117.148:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:00.303870 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.296710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:42673 (local address 127.12.117.190:38721)
0810 07:19:00.296836 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.296841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.297969 (+ 1128us) server_negotiation.cc:407] Connection header received
0810 07:19:00.299406 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.299410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.299465 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.299540 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.300397 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.300936 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.302075 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.302300 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.302386 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.303429 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.303434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.303436 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.303447 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.303466 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.303473 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.303531 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.303670 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.303730 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:00.304768 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.297169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38721 (local address 127.12.117.129:42673)
0810 07:19:00.297834 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.297873 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.299157 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.299715 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.299722 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.299802 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:00.300260 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.300273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.301056 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.301059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.301941 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.301950 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.302439 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.302443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.302601 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.303174 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:00.303176 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:00.303213 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:00.303300 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:00.303540 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:00.303550 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:00.303556 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.303661 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.303910 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":156,"threads_started":1}
I20260810 07:19:00.305914 11155 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38721
I20260810 07:19:00.306125 11155 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:00.306589 11155 heartbeater.cc:511] Master 127.12.117.190:38721 requested a full tablet report, sending...
I20260810 07:19:00.308147 10842 ts_manager.cc:194] Registered new tserver with Master: b2f71f82a7ce4c2080c971e7944c0a2b (127.12.117.129:37963)
I20260810 07:19:00.309686 10842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.129:42673
W20260810 07:19:00.405932 11159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:00.406158 11159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:00.406193 11159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:00.410017 11159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:00.410061 11159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:00.410116 11159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:19:00.414583 11159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:00.415474 11159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:00.416654 11159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:00.425357 11167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.425490 11159 server_base.cc:1034] running on GCE node
W20260810 07:19:00.425377 11172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.425381 11166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.425971 11159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:00.426544 11159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:00.427697 11159 hybrid_clock.cc:648] HybridClock initialized: now 1786346340427680 us; error 26 us; skew 500 ppm
I20260810 07:19:00.429383 11159 webserver.cc:460] Webserver started at http://127.12.117.130:45639/ using document root <none> and password file <none>
I20260810 07:19:00.429890 11159 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:00.429955 11159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:00.430147 11159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:00.432235 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "27d755e35d9c4a87acd3eb54dd061105"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f81dde4900fbd287916f602e1fda20d7"
server_key_iv: "591c9295d5a6a69634fef3c02744659e"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.432758 11159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "27d755e35d9c4a87acd3eb54dd061105"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f81dde4900fbd287916f602e1fda20d7"
server_key_iv: "591c9295d5a6a69634fef3c02744659e"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.436204 11159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:19:00.438377 11181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.439543 11159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 07:19:00.439646 11159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "27d755e35d9c4a87acd3eb54dd061105"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "f81dde4900fbd287916f602e1fda20d7"
server_key_iv: "591c9295d5a6a69634fef3c02744659e"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.439735 11159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:00.450253 11159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:00.450802 11159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:00.450956 11159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:00.451496 11159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:00.452339 11159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:00.452396 11159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.452447 11159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:00.452472 11159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.460647 11159 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:42671
I20260810 07:19:00.460668 11370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:42671 every 8 connection(s)
I20260810 07:19:00.461450 11159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:00.463960 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 11159
I20260810 07:19:00.464078 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:00.464382 12758 external_mini_cluster.cc:1428] Setting key d237f4632ad1f8adbb454a0435f00afd
I20260810 07:19:00.466513 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--builtin_ntp_servers=127.12.117.148:46149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:00.475485 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.463043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:43937 (local address 127.12.117.190:38721)
0810 07:19:00.463204 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.463209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.464478 (+ 1269us) server_negotiation.cc:407] Connection header received
0810 07:19:00.466268 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.466272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.466354 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.466454 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.467735 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.468686 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.470273 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.470537 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.470595 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.474933 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.474937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.474940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.474953 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.474976 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.474986 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.475050 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.475216 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.475304 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 07:19:00.476459 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.463535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38721 (local address 127.12.117.130:43937)
0810 07:19:00.464330 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.464384 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.466016 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.466691 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.466701 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.466818 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:00.467510 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.467532 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.468819 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.468823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.470152 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.470162 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.470747 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.470750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.470908 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.474673 (+ 3765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:00.474675 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:00.474716 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:00.474780 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:00.475074 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:00.475084 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:00.475090 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.475191 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.475502 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":142,"threads_started":1}
I20260810 07:19:00.477572 11371 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38721
I20260810 07:19:00.477792 11371 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:00.478281 11371 heartbeater.cc:511] Master 127.12.117.190:38721 requested a full tablet report, sending...
I20260810 07:19:00.479290 10842 ts_manager.cc:194] Registered new tserver with Master: 27d755e35d9c4a87acd3eb54dd061105 (127.12.117.130:42671)
I20260810 07:19:00.479853 10842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.130:43937
W20260810 07:19:00.577298 11375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:00.577517 11375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:00.577566 11375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:00.581430 11375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:00.581473 11375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:00.581542 11375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:19:00.586061 11375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:46149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:00.586920 11375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:00.588148 11375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:00.596647 11387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.596665 11385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:00.596666 11390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:00.596839 11375 server_base.cc:1034] running on GCE node
I20260810 07:19:00.597213 11375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:00.597762 11375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:00.598910 11375 hybrid_clock.cc:648] HybridClock initialized: now 1786346340598896 us; error 24 us; skew 500 ppm
I20260810 07:19:00.600569 11375 webserver.cc:460] Webserver started at http://127.12.117.131:33807/ using document root <none> and password file <none>
I20260810 07:19:00.601065 11375 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:00.601132 11375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:00.601308 11375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:00.603338 11375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "f70807997d114e998d304fae33abab4c"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "57cb32a7419fbde720c29af196acdbef"
server_key_iv: "af4aeeb211f6bca427cda6c7d502c0c8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.603868 11375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "f70807997d114e998d304fae33abab4c"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "57cb32a7419fbde720c29af196acdbef"
server_key_iv: "af4aeeb211f6bca427cda6c7d502c0c8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.607164 11375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:19:00.609323 11399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.610383 11375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:00.610483 11375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "f70807997d114e998d304fae33abab4c"
format_stamp: "Formatted at 2026-08-10 07:19:00 on dist-test-slave-gjrz"
server_key: "57cb32a7419fbde720c29af196acdbef"
server_key_iv: "af4aeeb211f6bca427cda6c7d502c0c8"
server_key_version: "encryptionkey@0"
I20260810 07:19:00.610567 11375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:00.628482 11375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:00.629019 11375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:00.629171 11375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:00.629676 11375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:00.630497 11375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:00.630544 11375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.630594 11375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:00.630622 11375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:00.638443 11375 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:43203
I20260810 07:19:00.638465 11587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:43203 every 8 connection(s)
I20260810 07:19:00.639204 11375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:00.644197 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 11375
I20260810 07:19:00.644310 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:00.644611 12758 external_mini_cluster.cc:1428] Setting key 7de1188d6bb597cd0ae8b0dbbc86f1c5
I20260810 07:19:00.647948 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.640827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:40111 (local address 127.12.117.190:38721)
0810 07:19:00.640959 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.640963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.641887 (+ 924us) server_negotiation.cc:407] Connection header received
0810 07:19:00.643185 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.643188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.643236 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.643338 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.644200 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.644935 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.646170 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.646392 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.646493 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.647569 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.647573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.647575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.647595 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.647615 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.647623 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.647679 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.647793 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.647842 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:00.648744 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.641164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38721 (local address 127.12.117.131:40111)
0810 07:19:00.641745 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.641781 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.642993 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.643516 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.643523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.643605 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:00.644045 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.644057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.645060 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.645063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.646064 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.646076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.646671 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.646674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.646820 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.647326 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:00.647327 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:00.647365 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:00.647426 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:00.647687 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:00.647693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:00.647698 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.647785 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.647993 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
I20260810 07:19:00.649715 11590 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38721
I20260810 07:19:00.649920 11590 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:00.650399 11590 heartbeater.cc:511] Master 127.12.117.190:38721 requested a full tablet report, sending...
I20260810 07:19:00.651355 10842 ts_manager.cc:194] Registered new tserver with Master: f70807997d114e998d304fae33abab4c (127.12.117.131:43203)
I20260810 07:19:00.651885 10842 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.131:40111
I20260810 07:19:00.658059 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:00.663228 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.659874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36662 (local address 127.12.117.190:38721)
0810 07:19:00.659977 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.659982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.660170 (+ 188us) server_negotiation.cc:407] Connection header received
0810 07:19:00.660270 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.660273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.660311 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.660371 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:00.660625 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.661089 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.661896 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.662100 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.662183 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.662877 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:00.662880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:00.662882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:00.662892 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:00.662908 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:00.662916 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:00.662972 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.663056 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.663107 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:19:00.665493 10842 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:00.668334 10842 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:00.681612 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.676069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.12.117.130:42671)
0810 07:19:00.677005 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.677016 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.677041 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:19:00.677119 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.677124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.677313 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.677607 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:00.677779 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.678897 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.680370 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.680993 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.681111 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.681282 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.681378 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.681465 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":123,"threads_started":1}
I20260810 07:19:00.681614 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.675889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:42671 (local address 127.0.0.1:53580)
0810 07:19:00.676197 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.676244 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.676357 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.677487 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.677494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.677533 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:00.677669 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.677676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.679006 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.679009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.680232 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.680241 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.681102 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.681106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.681207 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.681241 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.681350 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.681463 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":49,"spinlock_wait_cycles":1280,"thread_start_us":51,"threads_started":1}
I20260810 07:19:00.681782 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.676241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37963 (local address 127.0.0.1:32794)
0810 07:19:00.676679 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.676691 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.676772 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.677487 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.677494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.677526 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:00.677845 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.677851 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.679291 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.679295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.680202 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.680211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.681389 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.681393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.681479 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.681511 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.681570 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.681638 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":5,"spinlock_wait_cycles":3840,"thread_start_us":112,"threads_started":1}
I20260810 07:19:00.682149 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.676554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:43203 (local address 127.0.0.1:34796)
0810 07:19:00.676787 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.676800 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.676883 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.677703 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.677706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.677725 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:00.677822 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.677828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.678818 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.678822 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.680515 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.680523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.681483 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.681486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.681580 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.681599 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.681638 (+ 39us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.682032 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":31,"thread_start_us":90,"threads_started":1}
I20260810 07:19:00.682288 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.676671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32794 (local address 127.12.117.129:37963)
0810 07:19:00.677005 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.677045 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.677064 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:19:00.677127 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.677131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.677313 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.677985 (+ 672us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:00.678134 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.679160 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.680320 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.681051 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.681165 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.681321 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.682052 (+ 731us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.682144 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260810 07:19:00.682873 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.677015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.12.117.131:43203)
0810 07:19:00.677275 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.677286 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.677305 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:19:00.677371 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.677376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.677552 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.677938 (+ 386us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:00.678031 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.678710 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.680631 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.681363 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.682433 (+ 1070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.682599 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.682643 (+ 44us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.682727 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":79,"threads_started":1}
I20260810 07:19:00.684401 11256 tablet_service.cc:1467] Processing CreateTablet for tablet 39785b6980fd4efb8240bf35c42bae5b (DEFAULT_TABLE table=test-table [id=55ff3f2913b247bf8f9efa06d28c5988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:00.685303 11256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39785b6980fd4efb8240bf35c42bae5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:00.685210 11050 tablet_service.cc:1467] Processing CreateTablet for tablet 39785b6980fd4efb8240bf35c42bae5b (DEFAULT_TABLE table=test-table [id=55ff3f2913b247bf8f9efa06d28c5988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:00.685364 11478 tablet_service.cc:1467] Processing CreateTablet for tablet 39785b6980fd4efb8240bf35c42bae5b (DEFAULT_TABLE table=test-table [id=55ff3f2913b247bf8f9efa06d28c5988]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:00.686215 11050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39785b6980fd4efb8240bf35c42bae5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:00.686215 11478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39785b6980fd4efb8240bf35c42bae5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:00.690907 11614 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Bootstrap starting.
I20260810 07:19:00.692955 11614 tablet_bootstrap.cc:654] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:00.693053 11619 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Bootstrap starting.
I20260810 07:19:00.694280 11614 log.cc:824] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:00.694989 11620 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: Bootstrap starting.
I20260810 07:19:00.695132 11619 tablet_bootstrap.cc:654] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:00.695925 11619 log.cc:824] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:00.696610 11614 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: No bootstrap required, opened a new log
I20260810 07:19:00.696805 11614 ts_tablet_manager.cc:1397] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:19:00.697781 11619 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: No bootstrap required, opened a new log
I20260810 07:19:00.697958 11619 ts_tablet_manager.cc:1397] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:19:00.700790 11620 tablet_bootstrap.cc:654] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:00.700893 11614 raft_consensus.cc:348] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.701143 11614 raft_consensus.cc:374] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:00.701198 11614 raft_consensus.cc:729] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27d755e35d9c4a87acd3eb54dd061105, State: Initialized, Role: FOLLOWER
I20260810 07:19:00.701486 11620 log.cc:824] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:00.701646 11614 consensus_queue.cc:260] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.701843 11619 raft_consensus.cc:348] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.702106 11619 raft_consensus.cc:374] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:00.702172 11619 raft_consensus.cc:729] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70807997d114e998d304fae33abab4c, State: Initialized, Role: FOLLOWER
I20260810 07:19:00.702422 11371 heartbeater.cc:503] Master 127.12.117.190:38721 was elected leader, sending a full tablet report...
I20260810 07:19:00.702630 11614 ts_tablet_manager.cc:1428] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 07:19:00.702560 11619 consensus_queue.cc:260] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.703332 11620 tablet_bootstrap.cc:492] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: No bootstrap required, opened a new log
I20260810 07:19:00.703392 11590 heartbeater.cc:503] Master 127.12.117.190:38721 was elected leader, sending a full tablet report...
I20260810 07:19:00.703457 11619 ts_tablet_manager.cc:1428] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260810 07:19:00.703502 11620 ts_tablet_manager.cc:1397] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260810 07:19:00.707325 11620 raft_consensus.cc:348] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.707510 11620 raft_consensus.cc:374] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:00.707562 11620 raft_consensus.cc:729] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f71f82a7ce4c2080c971e7944c0a2b, State: Initialized, Role: FOLLOWER
I20260810 07:19:00.707890 11620 consensus_queue.cc:260] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.708421 11155 heartbeater.cc:503] Master 127.12.117.190:38721 was elected leader, sending a full tablet report...
I20260810 07:19:00.708648 11620 ts_tablet_manager.cc:1428] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260810 07:19:00.712242 11373 tablet.cc:2369] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:00.714037 11627 raft_consensus.cc:482] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:00.714156 11627 raft_consensus.cc:504] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.715117 11627 leader_election.cc:288] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27d755e35d9c4a87acd3eb54dd061105 (127.12.117.130:42671), f70807997d114e998d304fae33abab4c (127.12.117.131:43203)
I20260810 07:19:00.718618 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.715322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:42671 (local address 127.12.117.129:47971)
0810 07:19:00.715430 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.715444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.715518 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.715864 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.715866 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.715882 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:00.715987 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.715995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.716781 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.716785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.717606 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.717616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.718255 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.718259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.718354 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.718379 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.718447 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.718504 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 07:19:00.718657 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.715367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47971 (local address 127.12.117.130:42671)
0810 07:19:00.715466 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.715470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.715481 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:00.715718 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.715722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.715772 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.715849 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:00.716108 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.716686 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.717719 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.718151 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.718237 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.718272 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.718468 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.718533 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 07:19:00.718743 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.715633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45737 (local address 127.12.117.131:43203)
0810 07:19:00.715745 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.715750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.715931 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:19:00.716031 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.716034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.716077 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.716143 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:00.716369 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.716885 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.717860 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.718265 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.718369 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.718407 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.718587 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.718647 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 07:19:00.718750 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.715564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:43203 (local address 127.12.117.129:45737)
0810 07:19:00.715846 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.715859 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:00.715949 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:00.716159 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:00.716161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:00.716178 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:00.716283 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.716288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.716989 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.716992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.717747 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:00.717756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.718384 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.718387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:00.718469 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.718492 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:19:00.718563 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 07:19:00.718625 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
I20260810 07:19:00.719521 11509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39785b6980fd4efb8240bf35c42bae5b" candidate_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70807997d114e998d304fae33abab4c" is_pre_election: true
I20260810 07:19:00.719537 11290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39785b6980fd4efb8240bf35c42bae5b" candidate_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d755e35d9c4a87acd3eb54dd061105" is_pre_election: true
I20260810 07:19:00.719765 11509 raft_consensus.cc:2454] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2f71f82a7ce4c2080c971e7944c0a2b in term 0.
I20260810 07:19:00.719765 11290 raft_consensus.cc:2454] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2f71f82a7ce4c2080c971e7944c0a2b in term 0.
I20260810 07:19:00.720191 10965 leader_election.cc:302] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27d755e35d9c4a87acd3eb54dd061105, b2f71f82a7ce4c2080c971e7944c0a2b; no voters:
I20260810 07:19:00.720515 11627 raft_consensus.cc:2781] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:00.720611 11627 raft_consensus.cc:482] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:00.720667 11627 raft_consensus.cc:3037] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:00.721481 11627 raft_consensus.cc:504] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.721771 11627 leader_election.cc:288] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [CANDIDATE]: Term 1 election: Requested vote from peers 27d755e35d9c4a87acd3eb54dd061105 (127.12.117.130:42671), f70807997d114e998d304fae33abab4c (127.12.117.131:43203)
I20260810 07:19:00.722091 11290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39785b6980fd4efb8240bf35c42bae5b" candidate_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27d755e35d9c4a87acd3eb54dd061105"
I20260810 07:19:00.722147 11509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39785b6980fd4efb8240bf35c42bae5b" candidate_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70807997d114e998d304fae33abab4c"
I20260810 07:19:00.722196 11290 raft_consensus.cc:3037] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:00.722275 11509 raft_consensus.cc:3037] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:00.723148 11290 raft_consensus.cc:2454] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2f71f82a7ce4c2080c971e7944c0a2b in term 1.
I20260810 07:19:00.723158 11509 raft_consensus.cc:2454] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2f71f82a7ce4c2080c971e7944c0a2b in term 1.
I20260810 07:19:00.723444 10965 leader_election.cc:302] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27d755e35d9c4a87acd3eb54dd061105, b2f71f82a7ce4c2080c971e7944c0a2b; no voters:
I20260810 07:19:00.723618 11627 raft_consensus.cc:2781] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:00.723821 11627 raft_consensus.cc:686] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [term 1 LEADER]: Becoming Leader. State: Replica: b2f71f82a7ce4c2080c971e7944c0a2b, State: Running, Role: LEADER
I20260810 07:19:00.724076 11627 consensus_queue.cc:237] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } }
I20260810 07:19:00.726840 10836 catalog_manager.cc:5495] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b reported cstate change: term changed from 0 to 1, leader changed from <none> to b2f71f82a7ce4c2080c971e7944c0a2b (127.12.117.129). New cstate: current_term: 1 leader_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2f71f82a7ce4c2080c971e7944c0a2b" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 37963 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:00.737296 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.734426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.12.117.129:37963)
0810 07:19:00.734530 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:00.734533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:00.734545 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:00.734656 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:00.734658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:00.734704 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:00.734779 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:00.734956 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.735460 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.736210 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:00.736403 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:00.736462 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:00.736679 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:00.737015 (+ 336us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:00.737102 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:00.737126 (+ 24us) server_negotiation.cc:299] Negotiation successful
0810 07:19:00.737165 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:00.742123 11509 raft_consensus.cc:1261] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 1 FOLLOWER]: Refusing update from remote peer b2f71f82a7ce4c2080c971e7944c0a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:00.742161 11290 raft_consensus.cc:1261] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 1 FOLLOWER]: Refusing update from remote peer b2f71f82a7ce4c2080c971e7944c0a2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:00.742725 11627 consensus_queue.cc:1035] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f70807997d114e998d304fae33abab4c" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:00.742888 11630 consensus_queue.cc:1035] T 39785b6980fd4efb8240bf35c42bae5b P b2f71f82a7ce4c2080c971e7944c0a2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "27d755e35d9c4a87acd3eb54dd061105" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 42671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:00.748559 11637 mvcc.cc:205] Tried to move back new op lower bound from 7316874611673948160 to 7316874611608059904. Current Snapshot: MvccSnapshot[applied={T|T < 7316874611673948160}]
I20260810 07:19:00.749382 11638 mvcc.cc:205] Tried to move back new op lower bound from 7316874611673948160 to 7316874611608059904. Current Snapshot: MvccSnapshot[applied={T|T < 7316874611673948160}]
I20260810 07:19:00.757903 10838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:00.758020 10838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:00.760156 10838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa8dee124c0b4c3fadc349439e4ba5da: Sending DeleteTablet for 3 replicas of tablet 39785b6980fd4efb8240bf35c42bae5b
I20260810 07:19:00.760809 11256 tablet_service.cc:1514] Processing DeleteTablet for tablet 39785b6980fd4efb8240bf35c42bae5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:00 UTC) from {username='slave'} at 127.0.0.1:53580
I20260810 07:19:00.760910 11478 tablet_service.cc:1514] Processing DeleteTablet for tablet 39785b6980fd4efb8240bf35c42bae5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:00 UTC) from {username='slave'} at 127.0.0.1:34796
I20260810 07:19:00.761220 11649 tablet_replica.cc:330] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: stopping tablet replica
I20260810 07:19:00.761315 11651 tablet_replica.cc:330] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: stopping tablet replica
I20260810 07:19:00.761512 11651 raft_consensus.cc:2229] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:00.761504 11649 raft_consensus.cc:2229] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:00.761579 11050 tablet_service.cc:1514] Processing DeleteTablet for tablet 39785b6980fd4efb8240bf35c42bae5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:00 UTC) from {username='slave'} at 127.0.0.1:32794
I20260810 07:19:00.761761 11649 raft_consensus.cc:2258] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:00.761761 11651 raft_consensus.cc:2258] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:00.762414 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 10941
I20260810 07:19:00.770002 11651 ts_tablet_manager.cc:1905] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:00.772193 11651 ts_tablet_manager.cc:1918] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:00.772282 11651 log.cc:1196] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/39785b6980fd4efb8240bf35c42bae5b
I20260810 07:19:00.772518 11651 ts_tablet_manager.cc:1939] T 39785b6980fd4efb8240bf35c42bae5b P f70807997d114e998d304fae33abab4c: Deleting consensus metadata
I20260810 07:19:00.773675 10811 catalog_manager.cc:4841] TS f70807997d114e998d304fae33abab4c (127.12.117.131:43203): tablet 39785b6980fd4efb8240bf35c42bae5b (table test-table [id=55ff3f2913b247bf8f9efa06d28c5988]) successfully deleted
I20260810 07:19:00.774358 11649 ts_tablet_manager.cc:1905] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:00.776360 11649 ts_tablet_manager.cc:1918] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:00.776456 11649 log.cc:1196] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/39785b6980fd4efb8240bf35c42bae5b
I20260810 07:19:00.776748 11649 ts_tablet_manager.cc:1939] T 39785b6980fd4efb8240bf35c42bae5b P 27d755e35d9c4a87acd3eb54dd061105: Deleting consensus metadata
W20260810 07:19:00.776787 10806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:37963 (error 108)
I20260810 07:19:00.776882 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 11159
I20260810 07:19:00.779320 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.779009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:37963 (local address 127.0.0.1:32814)
0810 07:19:00.779158 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.779213 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:37963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260810 07:19:00.779445 10806 catalog_manager.cc:4584] TS b2f71f82a7ce4c2080c971e7944c0a2b (127.12.117.129:37963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 39785b6980fd4efb8240bf35c42bae5b: Network error: Client connection negotiation failed: client connection to 127.12.117.129:37963: connect: Connection refused (error 111)
I20260810 07:19:00.791432 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 11375
I20260810 07:19:00.797242 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:00.791521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:42671 (local address 127.0.0.1:53592)
0810 07:19:00.797105 (+ 5584us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:00.797161 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.130:42671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5497}
I20260810 07:19:00.803345 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 10776
2026-08-10T07:19:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (954 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:00 dist-test-slave-gjrz krb5kdc[11660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:00 dist-test-slave-gjrz krb5kdc[11660](info): set up 2 sockets
Aug 10 07:19:00 dist-test-slave-gjrz krb5kdc[11660](info): commencing operation
krb5kdc: starting...
W20260810 07:19:01.964886 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:02.079480 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38435
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:38435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:02.185695 11680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:02.185925 11680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:02.185961 11680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:02.189805 11680 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:02.189855 11680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:02.189872 11680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:02.189888 11680 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:02.189900 11680 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:02.194855 11680 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:38435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:38435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:02.195750 11680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:02.196657 11680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:02.204581 11689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.204667 11680 server_base.cc:1034] running on GCE node
W20260810 07:19:02.204586 11693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.204622 11691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.205269 11680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:02.206264 11680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:02.207412 11680 hybrid_clock.cc:648] HybridClock initialized: now 1786346342207397 us; error 26 us; skew 500 ppm
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:02.209931 11680 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:02.210748 11680 webserver.cc:460] Webserver started at http://127.12.117.190:46543/ using document root <none> and password file <none>
I20260810 07:19:02.211238 11680 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:02.211334 11680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:02.211513 11680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:02.213411 11680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "d869ca86d5dd4545b7d5db0e9860047c"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "024a508158d1fb2a9378dc7c93aada0c"
server_key_iv: "1bb0f7d9d6eb962fb89448b7c4739bf9"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.213912 11680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "d869ca86d5dd4545b7d5db0e9860047c"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "024a508158d1fb2a9378dc7c93aada0c"
server_key_iv: "1bb0f7d9d6eb962fb89448b7c4739bf9"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.217334 11680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:02.219570 11706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.220669 11680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:02.220763 11680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "d869ca86d5dd4545b7d5db0e9860047c"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "024a508158d1fb2a9378dc7c93aada0c"
server_key_iv: "1bb0f7d9d6eb962fb89448b7c4739bf9"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.220841 11680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:02.233566 11680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:02.236048 11680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:02.236184 11680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:02.242501 11680 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:38435
I20260810 07:19:02.242514 11797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:38435 every 8 connection(s)
I20260810 07:19:02.243399 11680 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:02.244664 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 11680
I20260810 07:19:02.244798 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:02.245132 12758 external_mini_cluster.cc:1428] Setting key 28607aab72fbd100b952f656b980f026
I20260810 07:19:02.246461 11798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:02.251945 11798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: Bootstrap starting.
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:02.254102 11798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:02.254900 11798 log.cc:824] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:02.256873 11798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: No bootstrap required, opened a new log
I20260810 07:19:02.258100 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.246463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.12.117.190:38435)
0810 07:19:02.246877 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.246891 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.246926 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:02.247637 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.247655 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.247995 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.248292 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.249078 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.250068 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.250905 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.251416 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.251560 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.253821 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.253826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.253838 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.253864 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.255490 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.255960 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.255964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.255969 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.256072 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.256349 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.256351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.256352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.256704 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.256837 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.257170 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.257441 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":141,"threads_started":1}
I20260810 07:19:02.261336 11798 raft_consensus.cc:348] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } }
I20260810 07:19:02.261533 11798 raft_consensus.cc:374] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:02.261581 11798 raft_consensus.cc:729] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d869ca86d5dd4545b7d5db0e9860047c, State: Initialized, Role: FOLLOWER
I20260810 07:19:02.261965 11798 consensus_queue.cc:260] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } }
I20260810 07:19:02.262087 11798 raft_consensus.cc:388] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:02.262148 11798 raft_consensus.cc:482] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:02.262241 11798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:02.262991 11798 raft_consensus.cc:504] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } }
I20260810 07:19:02.263327 11798 leader_election.cc:302] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d869ca86d5dd4545b7d5db0e9860047c; no voters:
I20260810 07:19:02.263605 11798 leader_election.cc:288] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:02.263715 11806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:02.263952 11806 raft_consensus.cc:686] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [term 1 LEADER]: Becoming Leader. State: Replica: d869ca86d5dd4545b7d5db0e9860047c, State: Running, Role: LEADER
I20260810 07:19:02.264266 11806 consensus_queue.cc:237] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } }
I20260810 07:19:02.264720 11798 sys_catalog.cc:564] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:02.265715 11809 sys_catalog.cc:455] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d869ca86d5dd4545b7d5db0e9860047c. Latest consensus state: current_term: 1 leader_uuid: "d869ca86d5dd4545b7d5db0e9860047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } } }
I20260810 07:19:02.265839 11809 sys_catalog.cc:458] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:02.266127 11808 sys_catalog.cc:455] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d869ca86d5dd4545b7d5db0e9860047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d869ca86d5dd4545b7d5db0e9860047c" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 38435 } } }
I20260810 07:19:02.266222 11808 sys_catalog.cc:458] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:02.266196 11822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:02.269295 11822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:02.276552 11822 catalog_manager.cc:1347] Generated new cluster ID: 1fc59913eb7b41ffa4c82dafc5a45597
I20260810 07:19:02.276621 11822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:02.288149 11822 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:02.288997 11822 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:02.297825 11822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: Generated new TSK 0
I20260810 07:19:02.298404 11822 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:19:02.328320 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:02.435807 11840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:02.436028 11840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:02.436072 11840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:02.439944 11840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:02.439994 11840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:02.440060 11840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:19:02.445130 11840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:02.446025 11840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:02.446954 11840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:02.455991 11849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.456066 11840 server_base.cc:1034] running on GCE node
W20260810 07:19:02.456092 11846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.456312 11845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.456655 11840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:02.457231 11840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:02.458384 11840 hybrid_clock.cc:648] HybridClock initialized: now 1786346342458366 us; error 31 us; skew 500 ppm
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:02.460889 11840 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:19:02.461673 11840 webserver.cc:460] Webserver started at http://127.12.117.129:38315/ using document root <none> and password file <none>
I20260810 07:19:02.462172 11840 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:02.462226 11840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:02.462386 11840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:02.464288 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "3cb3efaf7af54181901a41abb664f3a9"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "e946c006b53dc1c39d9f90f50b673ebb"
server_key_iv: "858e72ee747874beda336b6eb43f16b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.464771 11840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "3cb3efaf7af54181901a41abb664f3a9"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "e946c006b53dc1c39d9f90f50b673ebb"
server_key_iv: "858e72ee747874beda336b6eb43f16b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.468309 11840 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 07:19:02.470448 11858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.471585 11840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:02.471678 11840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "3cb3efaf7af54181901a41abb664f3a9"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "e946c006b53dc1c39d9f90f50b673ebb"
server_key_iv: "858e72ee747874beda336b6eb43f16b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.471755 11840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:02.487709 11840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:02.490193 11840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:02.490350 11840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:02.490909 11840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:02.491693 11840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:02.491739 11840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.491781 11840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:02.491809 11840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.499923 11840 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:33485
I20260810 07:19:02.499948 12040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:33485 every 8 connection(s)
I20260810 07:19:02.500721 11840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:02.504359 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 11840
I20260810 07:19:02.504477 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:02.504808 12758 external_mini_cluster.cc:1428] Setting key c36cea2c9f17ebe9b7b5badf214d1491
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:02.513484 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.502333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:34253 (local address 127.12.117.190:38435)
0810 07:19:02.502457 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.502461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.503210 (+ 749us) server_negotiation.cc:407] Connection header received
0810 07:19:02.504511 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.504515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.504571 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.504666 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.505895 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.506502 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.507676 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.507899 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.507987 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.510301 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.510304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.510306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.510330 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.511633 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.512094 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.512097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.512099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.512173 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.512475 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.512478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.512480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.512654 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.512740 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.513202 (+ 462us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.513326 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:19:02.514197 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.502670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38435 (local address 127.12.117.129:34253)
0810 07:19:02.503080 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.503118 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.504301 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.504802 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.504809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.505170 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:02.505747 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.505767 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.506635 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.506638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.507559 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.507569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.508032 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.508037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.508189 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.508673 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:02.508689 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:02.510151 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:02.511738 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.511744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.511757 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.511993 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.512256 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.512259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.512261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.512390 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.512754 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:02.512760 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:02.512982 (+ 222us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.513144 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.513439 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260810 07:19:02.515376 12043 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38435
I20260810 07:19:02.515589 12043 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:02.516016 12043 heartbeater.cc:511] Master 127.12.117.190:38435 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
I20260810 07:19:02.517546 11743 ts_manager.cc:194] Registered new tserver with Master: 3cb3efaf7af54181901a41abb664f3a9 (127.12.117.129:33485)
I20260810 07:19:02.518725 11743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:34253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:19:02.530252 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:02.637763 12049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:02.637985 12049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:02.638024 12049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:02.641913 12049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:02.641958 12049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:02.642009 12049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:19:02.646984 12049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:02.647912 12049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:02.648823 12049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:02.657824 12057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.657804 12060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.657819 12049 server_base.cc:1034] running on GCE node
W20260810 07:19:02.657846 12058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.658458 12049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:02.659034 12049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:02.660211 12049 hybrid_clock.cc:648] HybridClock initialized: now 1786346342660197 us; error 53 us; skew 500 ppm
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:02.663182 12049 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:19:02.663997 12049 webserver.cc:460] Webserver started at http://127.12.117.130:39513/ using document root <none> and password file <none>
I20260810 07:19:02.664535 12049 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:02.664595 12049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:02.664760 12049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:02.666740 12049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "fbc55f79828b4b9792d42ca919d0527e"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "72973d8385369a30b925bc582ac695b3"
server_key_iv: "38b7abe58fa1aa820bebaccc24c16d1c"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.667251 12049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "fbc55f79828b4b9792d42ca919d0527e"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "72973d8385369a30b925bc582ac695b3"
server_key_iv: "38b7abe58fa1aa820bebaccc24c16d1c"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.670666 12049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 07:19:02.672880 12075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.674146 12049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 07:19:02.674256 12049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "fbc55f79828b4b9792d42ca919d0527e"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "72973d8385369a30b925bc582ac695b3"
server_key_iv: "38b7abe58fa1aa820bebaccc24c16d1c"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.674338 12049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:02.690793 12049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:02.693521 12049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:02.693668 12049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:02.694171 12049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:02.694978 12049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:02.695024 12049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.695063 12049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:02.695084 12049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.703052 12049 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:38219
I20260810 07:19:02.703079 12254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:38219 every 8 connection(s)
I20260810 07:19:02.703886 12049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:02.705878 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12049
I20260810 07:19:02.706018 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:02.706305 12758 external_mini_cluster.cc:1428] Setting key 58bd17a9af1cb01a930f967200ecbf99
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:19:02.717864 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.705531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:33245 (local address 127.12.117.190:38435)
0810 07:19:02.705644 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.705647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.706454 (+ 807us) server_negotiation.cc:407] Connection header received
0810 07:19:02.707921 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.707924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.708012 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.708101 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.709166 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.709887 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.711477 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.711701 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.711781 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.714352 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.714356 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.714359 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.714388 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.715846 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.716335 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.716339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.716341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.716432 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.716794 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.716797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.716799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.717041 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.717153 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.717567 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.717692 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:02.718502 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.705839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38435 (local address 127.12.117.130:33245)
0810 07:19:02.706292 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.706342 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.707740 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.708243 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.708250 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.708563 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:02.709023 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.709036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.710021 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.710024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.711363 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.711375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.711845 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.711849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.712040 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.712588 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:02.712631 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:02.714183 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:02.715973 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.715978 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.715989 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.716233 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.716548 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.716551 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.716553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.716697 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.717134 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:02.717139 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:02.717350 (+ 211us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.717492 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.717764 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":125,"threads_started":1}
I20260810 07:19:02.719605 12257 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38435
I20260810 07:19:02.719838 12257 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
I20260810 07:19:02.720351 12257 heartbeater.cc:511] Master 127.12.117.190:38435 requested a full tablet report, sending...
I20260810 07:19:02.721418 11743 ts_manager.cc:194] Registered new tserver with Master: fbc55f79828b4b9792d42ca919d0527e (127.12.117.130:38219)
I20260810 07:19:02.722069 11743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:33245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:19:02.734325 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:02.843775 12268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:02.844020 12268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:02.844079 12268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:02.847978 12268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:02.848032 12268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:02.848107 12268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:19:02.853324 12268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:38435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:02.854398 12268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:02.855383 12268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:02.864727 12275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.864812 12268 server_base.cc:1034] running on GCE node
W20260810 07:19:02.864763 12274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:02.864765 12279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:02.865412 12268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:02.866113 12268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:02.867298 12268 hybrid_clock.cc:648] HybridClock initialized: now 1786346342867277 us; error 32 us; skew 500 ppm
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:02.869861 12268 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:19:02.870716 12268 webserver.cc:460] Webserver started at http://127.12.117.131:38619/ using document root <none> and password file <none>
I20260810 07:19:02.871240 12268 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:02.871344 12268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:02.871551 12268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:02.873481 12268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "b216687a076a49b8ba4f5b882dd90072"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "ad783c5d4f382e9dd9f2dc2ee5c670c3"
server_key_iv: "e0f74790bf470170902f06ecaf9c6f2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.874006 12268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "b216687a076a49b8ba4f5b882dd90072"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "ad783c5d4f382e9dd9f2dc2ee5c670c3"
server_key_iv: "e0f74790bf470170902f06ecaf9c6f2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.877591 12268 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:02.879912 12288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.881072 12268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:02.881176 12268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "b216687a076a49b8ba4f5b882dd90072"
format_stamp: "Formatted at 2026-08-10 07:19:02 on dist-test-slave-gjrz"
server_key: "ad783c5d4f382e9dd9f2dc2ee5c670c3"
server_key_iv: "e0f74790bf470170902f06ecaf9c6f2e"
server_key_version: "encryptionkey@0"
I20260810 07:19:02.881266 12268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:02.921929 12268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:02.925287 12268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:02.925486 12268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:02.926091 12268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:02.926954 12268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:02.927013 12268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.927068 12268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:02.927098 12268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:02.935457 12268 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:42035
I20260810 07:19:02.935488 12475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:42035 every 8 connection(s)
I20260810 07:19:02.936326 12268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:02.940873 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12268
I20260810 07:19:02.941013 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:02.941320 12758 external_mini_cluster.cc:1428] Setting key 87521677651204b7f3d8f604cfec5ae9
Aug 10 07:19:02 dist-test-slave-gjrz krb5kdc[11660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346342, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:02.949100 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.938022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:39607 (local address 127.12.117.190:38435)
0810 07:19:02.938165 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.938169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.938978 (+ 809us) server_negotiation.cc:407] Connection header received
0810 07:19:02.940256 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.940259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.940306 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.940378 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.941842 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.942449 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.943770 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.943986 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.944069 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.946247 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.946250 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.946252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.946276 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.947387 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.947822 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.947824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.947825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.947891 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.948136 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.948137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.948138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.948299 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.948372 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.948857 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.948968 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 07:19:02.949798 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.938390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:38435 (local address 127.12.117.131:39607)
0810 07:19:02.938829 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.938872 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.940061 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.940540 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.940547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.941033 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:02.941692 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.941711 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.942572 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.942575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.943663 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.943673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.944099 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.944101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.944234 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.944711 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:02.944726 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:02.946101 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:02.947501 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.947505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.947517 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.947752 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.947951 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:02.947953 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:02.947954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:02.948069 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:02.948381 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:02.948385 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:02.948606 (+ 221us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.948751 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.948983 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":106,"threads_started":1}
I20260810 07:19:02.950855 12477 heartbeater.cc:347] Connected to a master server at 127.12.117.190:38435
I20260810 07:19:02.951071 12477 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:02.951547 12477 heartbeater.cc:511] Master 127.12.117.190:38435 requested a full tablet report, sending...
I20260810 07:19:02.952581 11743 ts_manager.cc:194] Registered new tserver with Master: b216687a076a49b8ba4f5b882dd90072 (127.12.117.131:42035)
I20260810 07:19:02.953111 11743 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:39607
I20260810 07:19:02.955055 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:02.964071 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.956894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37980 (local address 127.12.117.190:38435)
0810 07:19:02.957004 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.957008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.957176 (+ 168us) server_negotiation.cc:407] Connection header received
0810 07:19:02.957286 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.957289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.957329 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.957391 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:02.958070 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.958561 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.959418 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.959636 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.959723 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.961183 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:02.961187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:02.961189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:02.961210 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:02.962236 (+ 1026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.962730 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.962732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.962734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.962806 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:02.963126 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:02.963129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:02.963131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:02.963396 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:02.963549 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.963830 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.963915 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:02.966923 11743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:02.969853 11743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:02.983765 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.977682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:33485 (local address 127.0.0.1:55968)
0810 07:19:02.978609 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.978653 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.978787 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.979543 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.979545 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.979572 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:02.979767 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.979775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.981199 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.981202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.982427 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.982435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983233 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.983363 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983401 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.983506 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.983621 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":165,"spinlock_wait_cycles":6272,"thread_start_us":74,"threads_started":1}
I20260810 07:19:02.983769 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.977797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.12.117.129:33485)
0810 07:19:02.978133 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.978149 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.978927 (+ 778us) server_negotiation.cc:407] Connection header received
0810 07:19:02.979008 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.979014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.979217 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.979396 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:02.979898 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.981060 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.982561 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983116 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983227 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983390 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.983539 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.983635 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":62,"threads_started":1}
I20260810 07:19:02.984120 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.977797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:42035 (local address 127.0.0.1:48914)
0810 07:19:02.978831 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.978844 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.978999 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.979530 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.979536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.979566 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:02.979767 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.979775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.981218 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.981220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.982331 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.982341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983644 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.983787 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983826 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.983894 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.983959 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":99,"spinlock_wait_cycles":9344,"thread_start_us":121,"threads_started":1}
I20260810 07:19:02.984127 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.978358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38219 (local address 127.0.0.1:44298)
0810 07:19:02.979208 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:02.979220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:02.979314 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:02.979796 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:02.979800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:02.979817 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:02.979920 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.979927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.980869 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.980873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.982416 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:02.982426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983644 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:02.983790 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983826 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 07:19:02.983894 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:02.983959 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":53,"thread_start_us":99,"threads_started":1}
I20260810 07:19:02.984261 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.978523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.12.117.130:38219)
0810 07:19:02.979029 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.979039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.979402 (+ 363us) server_negotiation.cc:407] Connection header received
0810 07:19:02.979494 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.979499 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.979669 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.979796 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:02.980024 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.980745 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.982522 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983342 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983461 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983580 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.984038 (+ 458us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.984134 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":119,"threads_started":1}
I20260810 07:19:02.984895 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:02.978112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48914 (local address 127.12.117.131:42035)
0810 07:19:02.978579 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:02.978590 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:02.978931 (+ 341us) server_negotiation.cc:407] Connection header received
0810 07:19:02.979158 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:02.979164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:02.979370 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:02.979517 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:02.979898 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.981066 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.982448 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:02.983342 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:02.983462 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:02.983580 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:02.984364 (+ 784us) server_negotiation.cc:299] Negotiation successful
0810 07:19:02.984511 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":139,"threads_started":1}
I20260810 07:19:02.987183 12369 tablet_service.cc:1467] Processing CreateTablet for tablet d122ff4618bc48e994dc551d65b2067f (DEFAULT_TABLE table=test-table [id=bee3540070954c7781c37e616329ddb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:02.987076 12151 tablet_service.cc:1467] Processing CreateTablet for tablet d122ff4618bc48e994dc551d65b2067f (DEFAULT_TABLE table=test-table [id=bee3540070954c7781c37e616329ddb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:02.987076 11938 tablet_service.cc:1467] Processing CreateTablet for tablet d122ff4618bc48e994dc551d65b2067f (DEFAULT_TABLE table=test-table [id=bee3540070954c7781c37e616329ddb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:02.988096 12369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d122ff4618bc48e994dc551d65b2067f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:02.988131 11938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d122ff4618bc48e994dc551d65b2067f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:02.988131 12151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d122ff4618bc48e994dc551d65b2067f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:02.993852 12510 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Bootstrap starting.
I20260810 07:19:02.995301 12514 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Bootstrap starting.
I20260810 07:19:02.995795 12512 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: Bootstrap starting.
I20260810 07:19:02.996155 12510 tablet_bootstrap.cc:654] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:02.997426 12514 tablet_bootstrap.cc:654] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:02.997419 12510 log.cc:824] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:02.997612 12512 tablet_bootstrap.cc:654] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:02.998567 12514 log.cc:824] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:02.998683 12512 log.cc:824] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:02.999555 12510 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: No bootstrap required, opened a new log
I20260810 07:19:02.999732 12510 ts_tablet_manager.cc:1397] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 07:19:03.000476 12512 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: No bootstrap required, opened a new log
I20260810 07:19:03.000485 12514 tablet_bootstrap.cc:492] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: No bootstrap required, opened a new log
I20260810 07:19:03.000659 12512 ts_tablet_manager.cc:1397] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:19:03.000659 12514 ts_tablet_manager.cc:1397] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 07:19:03.002745 12510 raft_consensus.cc:348] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.003123 12510 raft_consensus.cc:374] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:03.003221 12510 raft_consensus.cc:729] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b216687a076a49b8ba4f5b882dd90072, State: Initialized, Role: FOLLOWER
I20260810 07:19:03.003669 12510 consensus_queue.cc:260] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.004304 12477 heartbeater.cc:503] Master 127.12.117.190:38435 was elected leader, sending a full tablet report...
I20260810 07:19:03.004570 12510 ts_tablet_manager.cc:1428] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:03.004424 12514 raft_consensus.cc:348] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.004696 12514 raft_consensus.cc:374] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:03.004760 12514 raft_consensus.cc:729] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbc55f79828b4b9792d42ca919d0527e, State: Initialized, Role: FOLLOWER
I20260810 07:19:03.005220 12514 consensus_queue.cc:260] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.005228 12512 raft_consensus.cc:348] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.005486 12512 raft_consensus.cc:374] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:03.005559 12512 raft_consensus.cc:729] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cb3efaf7af54181901a41abb664f3a9, State: Initialized, Role: FOLLOWER
I20260810 07:19:03.005887 12257 heartbeater.cc:503] Master 127.12.117.190:38435 was elected leader, sending a full tablet report...
I20260810 07:19:03.006050 12512 consensus_queue.cc:260] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.006206 12514 ts_tablet_manager.cc:1428] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:19:03.006846 12043 heartbeater.cc:503] Master 127.12.117.190:38435 was elected leader, sending a full tablet report...
I20260810 07:19:03.006978 12512 ts_tablet_manager.cc:1428] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:19:03.073242 12526 raft_consensus.cc:482] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:03.073432 12526 raft_consensus.cc:504] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.074580 12526 leader_election.cc:288] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b216687a076a49b8ba4f5b882dd90072 (127.12.117.131:42035), fbc55f79828b4b9792d42ca919d0527e (127.12.117.130:38219)
I20260810 07:19:03.079772 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.074900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:47707 (local address 127.12.117.130:38219)
0810 07:19:03.075139 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.075143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.075523 (+ 380us) server_negotiation.cc:407] Connection header received
0810 07:19:03.075684 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.075687 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.075736 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.075806 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:03.076119 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.076828 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.078405 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.079056 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.079178 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.079234 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.079566 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.079644 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260810 07:19:03.079772 12506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.074900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:39381 (local address 127.12.117.131:42035)
0810 07:19:03.075038 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.075042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.075325 (+ 283us) server_negotiation.cc:407] Connection header received
0810 07:19:03.075545 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.075549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.075607 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.075685 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:03.075950 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.076768 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.078406 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.079056 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.079178 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.079234 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.079566 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.079644 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:19:03.079800 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.074786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:42035 (local address 127.12.117.129:39381)
0810 07:19:03.075203 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:03.075218 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:03.075425 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:03.075698 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:03.075701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:03.075715 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:03.075845 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.075853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.076872 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.076875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.078285 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.078297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.079203 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.079208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.079424 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.079445 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:19:03.079524 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 07:19:03.079619 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":117,"thread_start_us":108,"threads_started":1}
I20260810 07:19:03.079803 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.074792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:38219 (local address 127.12.117.129:47707)
0810 07:19:03.075450 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:03.075462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:03.075581 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:03.075873 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:03.075876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:03.075891 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:03.075993 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.075999 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.076935 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.076938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.078285 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.078297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.079203 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.079208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.079407 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.079444 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:19:03.079529 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:19:03.079619 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":98,"thread_start_us":121,"threads_started":1}
I20260810 07:19:03.080524 12184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d122ff4618bc48e994dc551d65b2067f" candidate_uuid: "3cb3efaf7af54181901a41abb664f3a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc55f79828b4b9792d42ca919d0527e" is_pre_election: true
I20260810 07:19:03.080829 12184 raft_consensus.cc:2454] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb3efaf7af54181901a41abb664f3a9 in term 0.
I20260810 07:19:03.080902 12407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d122ff4618bc48e994dc551d65b2067f" candidate_uuid: "3cb3efaf7af54181901a41abb664f3a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b216687a076a49b8ba4f5b882dd90072" is_pre_election: true
I20260810 07:19:03.081184 12407 raft_consensus.cc:2454] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3cb3efaf7af54181901a41abb664f3a9 in term 0.
I20260810 07:19:03.081336 11862 leader_election.cc:302] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb3efaf7af54181901a41abb664f3a9, fbc55f79828b4b9792d42ca919d0527e; no voters:
I20260810 07:19:03.081614 12526 raft_consensus.cc:2781] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:03.081702 12526 raft_consensus.cc:482] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:03.081753 12526 raft_consensus.cc:3037] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:03.082754 12526 raft_consensus.cc:504] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.083055 12526 leader_election.cc:288] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [CANDIDATE]: Term 1 election: Requested vote from peers b216687a076a49b8ba4f5b882dd90072 (127.12.117.131:42035), fbc55f79828b4b9792d42ca919d0527e (127.12.117.130:38219)
I20260810 07:19:03.083392 12407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d122ff4618bc48e994dc551d65b2067f" candidate_uuid: "3cb3efaf7af54181901a41abb664f3a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b216687a076a49b8ba4f5b882dd90072"
I20260810 07:19:03.083426 12184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d122ff4618bc48e994dc551d65b2067f" candidate_uuid: "3cb3efaf7af54181901a41abb664f3a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbc55f79828b4b9792d42ca919d0527e"
I20260810 07:19:03.083501 12407 raft_consensus.cc:3037] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:03.083531 12184 raft_consensus.cc:3037] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:03.084491 12184 raft_consensus.cc:2454] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb3efaf7af54181901a41abb664f3a9 in term 1.
I20260810 07:19:03.084568 12407 raft_consensus.cc:2454] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3cb3efaf7af54181901a41abb664f3a9 in term 1.
I20260810 07:19:03.084813 11862 leader_election.cc:302] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3cb3efaf7af54181901a41abb664f3a9, fbc55f79828b4b9792d42ca919d0527e; no voters:
I20260810 07:19:03.085007 12526 raft_consensus.cc:2781] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:03.085206 12526 raft_consensus.cc:686] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 1 LEADER]: Becoming Leader. State: Replica: 3cb3efaf7af54181901a41abb664f3a9, State: Running, Role: LEADER
I20260810 07:19:03.085527 12526 consensus_queue.cc:237] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } }
I20260810 07:19:03.088550 11743 catalog_manager.cc:5495] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3cb3efaf7af54181901a41abb664f3a9 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "3cb3efaf7af54181901a41abb664f3a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cb3efaf7af54181901a41abb664f3a9" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 33485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:03.107223 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.103905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.12.117.129:33485)
0810 07:19:03.104017 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.104020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.104032 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:03.104151 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.104153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.104203 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.104281 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:03.104470 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.104944 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.105810 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.106066 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.106165 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.106499 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:03.106893 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:03.107028 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.107089 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.107121 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:03.112591 12407 raft_consensus.cc:1261] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 1 FOLLOWER]: Refusing update from remote peer 3cb3efaf7af54181901a41abb664f3a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:03.112598 12184 raft_consensus.cc:1261] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 1 FOLLOWER]: Refusing update from remote peer 3cb3efaf7af54181901a41abb664f3a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:03.113242 12536 consensus_queue.cc:1035] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b216687a076a49b8ba4f5b882dd90072" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 42035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:03.113404 12526 consensus_queue.cc:1035] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbc55f79828b4b9792d42ca919d0527e" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:03.118909 12541 mvcc.cc:205] Tried to move back new op lower bound from 7316874621382967296 to 7316874621280931840. Current Snapshot: MvccSnapshot[applied={T|T < 7316874621382967296}]
I20260810 07:19:03.128527 11739 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:03.128677 11739 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:03.130946 11739 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d869ca86d5dd4545b7d5db0e9860047c: Sending DeleteTablet for 3 replicas of tablet d122ff4618bc48e994dc551d65b2067f
I20260810 07:19:03.131731 11938 tablet_service.cc:1514] Processing DeleteTablet for tablet d122ff4618bc48e994dc551d65b2067f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:55968
I20260810 07:19:03.131733 12151 tablet_service.cc:1514] Processing DeleteTablet for tablet d122ff4618bc48e994dc551d65b2067f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:44298
I20260810 07:19:03.132241 12555 tablet_replica.cc:330] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: stopping tablet replica
I20260810 07:19:03.132241 12369 tablet_service.cc:1514] Processing DeleteTablet for tablet d122ff4618bc48e994dc551d65b2067f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:03 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:48914
I20260810 07:19:03.132476 12556 tablet_replica.cc:330] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9: stopping tablet replica
I20260810 07:19:03.132753 12556 raft_consensus.cc:2229] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:03.132850 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 11840
I20260810 07:19:03.132901 12559 tablet_replica.cc:330] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: stopping tablet replica
I20260810 07:19:03.133100 12556 raft_consensus.cc:2258] T d122ff4618bc48e994dc551d65b2067f P 3cb3efaf7af54181901a41abb664f3a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:03.133147 12559 raft_consensus.cc:2229] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:03.133543 12559 raft_consensus.cc:2258] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:03.133548 12555 raft_consensus.cc:2229] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:03.133764 12555 raft_consensus.cc:2258] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:03.136701 12555 ts_tablet_manager.cc:1905] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:03.136714 12559 ts_tablet_manager.cc:1905] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:03.138684 12555 ts_tablet_manager.cc:1918] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:03.138748 12555 log.cc:1196] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/d122ff4618bc48e994dc551d65b2067f
I20260810 07:19:03.138957 12555 ts_tablet_manager.cc:1939] T d122ff4618bc48e994dc551d65b2067f P fbc55f79828b4b9792d42ca919d0527e: Deleting consensus metadata
I20260810 07:19:03.139186 12559 ts_tablet_manager.cc:1918] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:03.139318 12559 log.cc:1196] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/d122ff4618bc48e994dc551d65b2067f
I20260810 07:19:03.139643 12559 ts_tablet_manager.cc:1939] T d122ff4618bc48e994dc551d65b2067f P b216687a076a49b8ba4f5b882dd90072: Deleting consensus metadata
I20260810 07:19:03.140249 11709 catalog_manager.cc:4841] TS fbc55f79828b4b9792d42ca919d0527e (127.12.117.130:38219): tablet d122ff4618bc48e994dc551d65b2067f (table test-table [id=bee3540070954c7781c37e616329ddb3]) successfully deleted
I20260810 07:19:03.140938 11712 catalog_manager.cc:4841] TS b216687a076a49b8ba4f5b882dd90072 (127.12.117.131:42035): tablet d122ff4618bc48e994dc551d65b2067f (table test-table [id=bee3540070954c7781c37e616329ddb3]) successfully deleted
W20260810 07:19:03.143507 11709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:33485 (error 108)
I20260810 07:19:03.144338 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12049
I20260810 07:19:03.144519 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.144220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:33485 (local address 127.0.0.1:55984)
0810 07:19:03.144368 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:03.144439 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:33485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 07:19:03.164876 11709 catalog_manager.cc:4584] TS 3cb3efaf7af54181901a41abb664f3a9 (127.12.117.129:33485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d122ff4618bc48e994dc551d65b2067f: Network error: Client connection negotiation failed: client connection to 127.12.117.129:33485: connect: Connection refused (error 111)
I20260810 07:19:03.170037 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12268
I20260810 07:19:03.179570 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 11680
2026-08-10T07:19:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2376 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T07:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:03Z Disabled control of system clock
I20260810 07:19:03.213501 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37083
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:19:03.319401 12564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:03.319619 12564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:03.319653 12564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:03.323514 12564 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:03.323561 12564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:03.323575 12564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:03.323590 12564 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:03.323611 12564 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:03.328234 12564 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:03.329128 12564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:03.330350 12564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:03.338277 12573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.338353 12564 server_base.cc:1034] running on GCE node
W20260810 07:19:03.338305 12571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.338297 12576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.338982 12564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:03.339999 12564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:03.341156 12564 hybrid_clock.cc:648] HybridClock initialized: now 1786346343341141 us; error 24 us; skew 500 ppm
I20260810 07:19:03.342772 12564 webserver.cc:460] Webserver started at http://127.12.117.190:44099/ using document root <none> and password file <none>
I20260810 07:19:03.343345 12564 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:03.343417 12564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:03.343618 12564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:03.345481 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "59930fadaf8548959e7fec59cef94506"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "77e73d52d943daf2fe3ed5daeabdc174"
server_key_iv: "095b93f722585cb9b6bbc05f59f88634"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.345989 12564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "59930fadaf8548959e7fec59cef94506"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "77e73d52d943daf2fe3ed5daeabdc174"
server_key_iv: "095b93f722585cb9b6bbc05f59f88634"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.349329 12564 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:03.351460 12585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.352550 12564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:03.352667 12564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "59930fadaf8548959e7fec59cef94506"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "77e73d52d943daf2fe3ed5daeabdc174"
server_key_iv: "095b93f722585cb9b6bbc05f59f88634"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.352753 12564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:03.381366 12564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:03.381992 12564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:03.382162 12564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:03.388392 12564 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37083
I20260810 07:19:03.388406 12677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37083 every 8 connection(s)
I20260810 07:19:03.389245 12564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:03.392143 12684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:03.397233 12684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: Bootstrap starting.
I20260810 07:19:03.398335 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12564
I20260810 07:19:03.398465 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:03.398762 12758 external_mini_cluster.cc:1428] Setting key 5dcd1778f369f0d8d414fff0c097eb5e
I20260810 07:19:03.399919 12684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:03.400630 12684 log.cc:824] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:03.402570 12684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: No bootstrap required, opened a new log
I20260810 07:19:03.406559 12684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } }
I20260810 07:19:03.406780 12684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:03.406836 12684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59930fadaf8548959e7fec59cef94506, State: Initialized, Role: FOLLOWER
I20260810 07:19:03.407228 12684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } }
I20260810 07:19:03.407382 12684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:03.407444 12684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:03.407534 12684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:03.407536 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.399829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.12.117.190:37083)
0810 07:19:03.400393 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.400407 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.400439 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:03.401099 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.401123 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.401511 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.401785 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:03.402152 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.403114 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.404332 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.404832 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.404943 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.405760 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:03.405765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:03.405777 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:03.406132 (+ 355us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:03.406185 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:03.406204 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:03.406272 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.406429 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.406693 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":116,"threads_started":1}
I20260810 07:19:03.408354 12684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } }
I20260810 07:19:03.408695 12684 leader_election.cc:302] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59930fadaf8548959e7fec59cef94506; no voters:
I20260810 07:19:03.409003 12684 leader_election.cc:288] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:03.409081 12696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:03.409279 12696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [term 1 LEADER]: Becoming Leader. State: Replica: 59930fadaf8548959e7fec59cef94506, State: Running, Role: LEADER
I20260810 07:19:03.409641 12696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } }
I20260810 07:19:03.410100 12684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:03.411151 12698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59930fadaf8548959e7fec59cef94506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } } }
I20260810 07:19:03.411296 12698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:03.411586 12699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59930fadaf8548959e7fec59cef94506. Latest consensus state: current_term: 1 leader_uuid: "59930fadaf8548959e7fec59cef94506" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59930fadaf8548959e7fec59cef94506" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37083 } } }
I20260810 07:19:03.411691 12699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:03.411672 12710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:03.414459 12710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:03.418968 12710 catalog_manager.cc:1347] Generated new cluster ID: 8223d0e82a014468b05091279d519d4c
I20260810 07:19:03.419023 12710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:03.427582 12710 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:03.428709 12710 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:03.438861 12710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: Generated new TSK 0
I20260810 07:19:03.439586 12710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:03.449774 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--builtin_ntp_servers=127.12.117.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:19:03.563390 12722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:03.563609 12722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:03.563659 12722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:03.567555 12722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:03.567600 12722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:03.567670 12722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:19:03.572176 12722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:03.573064 12722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:03.574302 12722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:03.583170 12730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.583166 12731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.583237 12722 server_base.cc:1034] running on GCE node
W20260810 07:19:03.583161 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.583791 12722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:03.584339 12722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:03.585489 12722 hybrid_clock.cc:648] HybridClock initialized: now 1786346343585467 us; error 32 us; skew 500 ppm
I20260810 07:19:03.587141 12722 webserver.cc:460] Webserver started at http://127.12.117.129:41727/ using document root <none> and password file <none>
I20260810 07:19:03.587687 12722 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:03.587749 12722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:03.587942 12722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:03.589797 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "d197569ddb4c441a9b1d51bb5365b765"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "9fe56b8103566d16cf7d883d69603c71"
server_key_iv: "ef4f598cc2d8396486b8b36b0f3f0dc8"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.590294 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "d197569ddb4c441a9b1d51bb5365b765"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "9fe56b8103566d16cf7d883d69603c71"
server_key_iv: "ef4f598cc2d8396486b8b36b0f3f0dc8"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.593611 12722 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:03.595877 12740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.596952 12722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:03.597045 12722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "d197569ddb4c441a9b1d51bb5365b765"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "9fe56b8103566d16cf7d883d69603c71"
server_key_iv: "ef4f598cc2d8396486b8b36b0f3f0dc8"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.597139 12722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:03.659116 12722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:03.659837 12722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:03.660000 12722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:03.660575 12722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:03.661501 12722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:03.661548 12722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.661602 12722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:03.661630 12722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.669828 12722 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:46339
I20260810 07:19:03.669857 12923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:46339 every 8 connection(s)
I20260810 07:19:03.670660 12722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:03.676782 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12722
I20260810 07:19:03.676895 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:03.677201 12758 external_mini_cluster.cc:1428] Setting key b5cf41ab297c473ce557a217434a165b
I20260810 07:19:03.679189 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--builtin_ntp_servers=127.12.117.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:03.679734 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.672344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45991 (local address 127.12.117.190:37083)
0810 07:19:03.672475 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.672484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.673478 (+ 994us) server_negotiation.cc:407] Connection header received
0810 07:19:03.674820 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.674823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.674879 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.674952 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:03.675880 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.676450 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.677707 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.677987 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.678076 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.679223 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:03.679226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:03.679229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:03.679242 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:03.679287 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:03.679299 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:03.679373 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.679573 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.679627 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:03.680735 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.672714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37083 (local address 127.12.117.129:45991)
0810 07:19:03.673338 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:03.673377 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:03.674611 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:03.675142 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:03.675149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:03.675232 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:03.675739 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.675753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.676593 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.676596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.677579 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.677593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.678139 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.678143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.678331 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.678964 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:03.678966 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:03.679004 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:03.679066 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:03.679418 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:03.679428 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:03.679434 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:03.679551 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 07:19:03.679828 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":118,"threads_started":1}
I20260810 07:19:03.681962 12924 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37083
I20260810 07:19:03.682200 12924 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:03.682762 12924 heartbeater.cc:511] Master 127.12.117.190:37083 requested a full tablet report, sending...
I20260810 07:19:03.684219 12611 ts_manager.cc:194] Registered new tserver with Master: d197569ddb4c441a9b1d51bb5365b765 (127.12.117.129:46339)
I20260810 07:19:03.685336 12611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.129:45991
W20260810 07:19:03.788926 12927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:03.789150 12927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:03.789188 12927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:03.793081 12927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:03.793123 12927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:03.793185 12927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:19:03.797758 12927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:03.798653 12927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:03.799902 12927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:03.808818 12936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.808790 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.808954 12927 server_base.cc:1034] running on GCE node
W20260810 07:19:03.808802 12941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.809445 12927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:03.810072 12927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:03.811226 12927 hybrid_clock.cc:648] HybridClock initialized: now 1786346343811207 us; error 29 us; skew 500 ppm
I20260810 07:19:03.812950 12927 webserver.cc:460] Webserver started at http://127.12.117.130:41601/ using document root <none> and password file <none>
I20260810 07:19:03.813468 12927 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:03.813534 12927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:03.813719 12927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:03.815656 12927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "b17bec14bc8e465283b7fa6412bb9fe3"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "0dc1a055856e326671ee091aba2d00e1"
server_key_iv: "8ed8018ec2ef5b6c77a8ee4e3f935d1a"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.816167 12927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "b17bec14bc8e465283b7fa6412bb9fe3"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "0dc1a055856e326671ee091aba2d00e1"
server_key_iv: "8ed8018ec2ef5b6c77a8ee4e3f935d1a"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.819497 12927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:03.821683 12952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.822759 12927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:03.822870 12927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "b17bec14bc8e465283b7fa6412bb9fe3"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "0dc1a055856e326671ee091aba2d00e1"
server_key_iv: "8ed8018ec2ef5b6c77a8ee4e3f935d1a"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.822956 12927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:03.834877 12927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:03.835470 12927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:03.835630 12927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:03.836164 12927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:03.837001 12927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:03.837047 12927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.837096 12927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:03.837126 12927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:03.845191 12927 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:39607
I20260810 07:19:03.845253 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:39607 every 8 connection(s)
I20260810 07:19:03.846041 12927 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:03.854532 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.847692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47055 (local address 127.12.117.190:37083)
0810 07:19:03.847811 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:03.847815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:03.848800 (+ 985us) server_negotiation.cc:407] Connection header received
0810 07:19:03.850124 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:03.850128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:03.850183 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:03.850259 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:03.851124 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.851708 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.852817 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.853066 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.853162 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.854084 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:03.854088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:03.854091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:03.854104 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:03.854124 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:03.854131 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:03.854189 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:03.854322 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:19:03.854388 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:03.855322 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:03.848045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37083 (local address 127.12.117.130:47055)
0810 07:19:03.848666 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:03.848706 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:03.849919 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:03.850427 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:03.850433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:03.850514 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:03.850984 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.850998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.851828 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.851831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.852701 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:03.852710 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:03.853206 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:03.853210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:03.853368 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:03.853844 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:03.853846 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:03.853882 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:03.853940 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:03.854198 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:03.854207 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:03.854213 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:03.854313 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:19:03.854564 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":116,"threads_started":1}
I20260810 07:19:03.856295 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 12927
I20260810 07:19:03.856418 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:03.856410 13149 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37083
I20260810 07:19:03.856611 13149 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:03.856716 12758 external_mini_cluster.cc:1428] Setting key 27eb8a7faf44184c5bc4233090072acb
I20260810 07:19:03.857105 13149 heartbeater.cc:511] Master 127.12.117.190:37083 requested a full tablet report, sending...
I20260810 07:19:03.858112 12611 ts_manager.cc:194] Registered new tserver with Master: b17bec14bc8e465283b7fa6412bb9fe3 (127.12.117.130:39607)
I20260810 07:19:03.858557 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--builtin_ntp_servers=127.12.117.148:41633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:03.858662 12611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.130:47055
W20260810 07:19:03.969296 13153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:03.969532 13153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:03.969568 13153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:03.973565 13153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:03.973618 13153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:03.973668 13153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:19:03.978288 13153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:41633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:03.979179 13153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:03.980463 13153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:03.989506 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:03.989506 13161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.989620 13153 server_base.cc:1034] running on GCE node
W20260810 07:19:03.989567 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:03.990134 13153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:03.990741 13153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:03.991936 13153 hybrid_clock.cc:648] HybridClock initialized: now 1786346343991917 us; error 30 us; skew 500 ppm
I20260810 07:19:03.993675 13153 webserver.cc:460] Webserver started at http://127.12.117.131:37869/ using document root <none> and password file <none>
I20260810 07:19:03.994222 13153 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:03.994292 13153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:03.994481 13153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:03.996486 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "7d025eb828c44e9999a54a081d436511"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "711b7f88697c73f4c2d784169e688beb"
server_key_iv: "2ff39abc6cd7a53df3a72dfb292e23c9"
server_key_version: "encryptionkey@0"
I20260810 07:19:03.997025 13153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "7d025eb828c44e9999a54a081d436511"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "711b7f88697c73f4c2d784169e688beb"
server_key_iv: "2ff39abc6cd7a53df3a72dfb292e23c9"
server_key_version: "encryptionkey@0"
I20260810 07:19:04.000583 13153 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:04.002974 13174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:04.004168 13153 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:04.004280 13153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "7d025eb828c44e9999a54a081d436511"
format_stamp: "Formatted at 2026-08-10 07:19:03 on dist-test-slave-gjrz"
server_key: "711b7f88697c73f4c2d784169e688beb"
server_key_iv: "2ff39abc6cd7a53df3a72dfb292e23c9"
server_key_version: "encryptionkey@0"
I20260810 07:19:04.004377 13153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:04.017623 13153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:04.018230 13153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:04.018396 13153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:04.018929 13153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:04.019846 13153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:04.019894 13153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:04.019944 13153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:04.019974 13153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:04.028342 13153 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:46519
I20260810 07:19:04.028362 13378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:46519 every 8 connection(s)
I20260810 07:19:04.029199 13153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:04.034838 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13153
I20260810 07:19:04.034933 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:04.035193 12758 external_mini_cluster.cc:1428] Setting key 5b3155a2435659dee8fdae3cb442a1c1
I20260810 07:19:04.038249 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.030935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:40875 (local address 127.12.117.190:37083)
0810 07:19:04.031060 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.031064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.032112 (+ 1048us) server_negotiation.cc:407] Connection header received
0810 07:19:04.033474 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.033477 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.033535 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.033615 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:04.034492 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.035085 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.036432 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.036722 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.036806 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.037887 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:04.037890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:04.037892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:04.037904 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:04.037924 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:04.037932 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:04.037988 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.038090 (+ 102us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.038142 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:04.039060 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.031358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37083 (local address 127.12.117.131:40875)
0810 07:19:04.031973 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.032010 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.033263 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.033784 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.033792 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.033877 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:04.034348 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.034361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.035207 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.035209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.036286 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.036301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.036868 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.036872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.037080 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.037652 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:04.037654 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:04.037691 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:04.037753 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:04.037992 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:04.037997 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:04.038001 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.038096 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.038313 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":123,"threads_started":1}
I20260810 07:19:04.040033 13381 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37083
I20260810 07:19:04.040300 13381 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:04.040803 13381 heartbeater.cc:511] Master 127.12.117.190:37083 requested a full tablet report, sending...
I20260810 07:19:04.041785 12611 ts_manager.cc:194] Registered new tserver with Master: 7d025eb828c44e9999a54a081d436511 (127.12.117.131:46519)
I20260810 07:19:04.042326 12611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.131:40875
I20260810 07:19:04.048796 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:04.054261 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.050609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.12.117.190:37083)
0810 07:19:04.050737 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.050745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.050910 (+ 165us) server_negotiation.cc:407] Connection header received
0810 07:19:04.051004 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.051006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.051044 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.051119 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:04.051381 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.051879 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.052769 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.052968 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.053037 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.053837 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:04.053840 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:04.053842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:04.053854 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:04.053873 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:04.053880 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:04.053978 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.054123 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.054157 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:04.056932 12611 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:04.059782 12611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:04.072928 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.067526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:39607 (local address 127.0.0.1:38776)
0810 07:19:04.068323 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.068337 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.068424 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.069138 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.069141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.069157 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.069240 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.069245 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.070406 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.070410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.071561 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.071570 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.072424 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.072428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.072528 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.072573 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.072675 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.072789 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":36,"thread_start_us":93,"threads_started":1}
I20260810 07:19:04.072948 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.068138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.12.117.130:39607)
0810 07:19:04.068677 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.068689 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.068714 (+ 25us) server_negotiation.cc:407] Connection header received
0810 07:19:04.068787 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.068792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.068977 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.069353 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.069457 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.070292 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.071699 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.072306 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.072419 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.072587 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.072698 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.072788 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":136,"threads_started":1}
I20260810 07:19:04.073184 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.067386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:46519 (local address 127.0.0.1:46234)
0810 07:19:04.067893 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.067943 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.068066 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.068803 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.068809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.068834 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.068960 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.068967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.070275 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.070278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.071486 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.071494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.072827 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.072831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.072923 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.072945 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.073010 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.073078 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":187,"thread_start_us":67,"threads_started":1}
I20260810 07:19:04.073643 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.067527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46234 (local address 127.12.117.131:46519)
0810 07:19:04.067976 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.067988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.068012 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:04.068280 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.068287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.068490 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.068636 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.069105 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.070169 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.071626 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.072496 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.072613 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.072767 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.073271 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.073367 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":152,"threads_started":1}
I20260810 07:19:04.073740 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.068214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46339 (local address 127.0.0.1:53958)
0810 07:19:04.068994 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.069006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.069080 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.069726 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.069729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.069747 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.069821 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.069826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.071102 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.071105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.072185 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.072195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.073363 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.073367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.073453 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.073476 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.073541 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.073619 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":6,"thread_start_us":95,"threads_started":1}
I20260810 07:19:04.073824 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.068627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.12.117.129:46339)
0810 07:19:04.069304 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.069314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.069337 (+ 23us) server_negotiation.cc:407] Connection header received
0810 07:19:04.069407 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.069411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.069587 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.069720 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.069929 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.070976 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.072312 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.073057 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.073168 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.073295 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.073597 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.073683 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":116,"threads_started":1}
I20260810 07:19:04.076190 13032 tablet_service.cc:1467] Processing CreateTablet for tablet 6e25119ae807427e82c8ef0365745506 (DEFAULT_TABLE table=test-table [id=2686ae617f034e3585066177adc036f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:04.076434 13275 tablet_service.cc:1467] Processing CreateTablet for tablet 6e25119ae807427e82c8ef0365745506 (DEFAULT_TABLE table=test-table [id=2686ae617f034e3585066177adc036f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:04.077137 13032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e25119ae807427e82c8ef0365745506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:04.076797 12833 tablet_service.cc:1467] Processing CreateTablet for tablet 6e25119ae807427e82c8ef0365745506 (DEFAULT_TABLE table=test-table [id=2686ae617f034e3585066177adc036f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:04.077457 13275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e25119ae807427e82c8ef0365745506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:04.077695 12833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e25119ae807427e82c8ef0365745506. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:04.083742 13410 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Bootstrap starting.
I20260810 07:19:04.084271 13411 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Bootstrap starting.
I20260810 07:19:04.085108 13412 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Bootstrap starting.
I20260810 07:19:04.086015 13410 tablet_bootstrap.cc:654] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:04.086382 13411 tablet_bootstrap.cc:654] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:04.086746 13412 tablet_bootstrap.cc:654] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:04.086889 13410 log.cc:824] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:04.087145 13411 log.cc:824] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:04.087469 13412 log.cc:824] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:04.088963 13410 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: No bootstrap required, opened a new log
I20260810 07:19:04.089011 13411 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: No bootstrap required, opened a new log
I20260810 07:19:04.089169 13410 ts_tablet_manager.cc:1397] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 07:19:04.089200 13411 ts_tablet_manager.cc:1397] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:19:04.089291 13412 tablet_bootstrap.cc:492] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: No bootstrap required, opened a new log
I20260810 07:19:04.089442 13412 ts_tablet_manager.cc:1397] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 07:19:04.092272 13411 raft_consensus.cc:348] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.092479 13411 raft_consensus.cc:374] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:04.092549 13411 raft_consensus.cc:729] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d197569ddb4c441a9b1d51bb5365b765, State: Initialized, Role: FOLLOWER
I20260810 07:19:04.092952 13411 consensus_queue.cc:260] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.093318 13412 raft_consensus.cc:348] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.093318 13410 raft_consensus.cc:348] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.093556 13410 raft_consensus.cc:374] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:04.093550 12924 heartbeater.cc:503] Master 127.12.117.190:37083 was elected leader, sending a full tablet report...
I20260810 07:19:04.093650 13412 raft_consensus.cc:374] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:04.093663 13410 raft_consensus.cc:729] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17bec14bc8e465283b7fa6412bb9fe3, State: Initialized, Role: FOLLOWER
I20260810 07:19:04.093765 13412 raft_consensus.cc:729] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d025eb828c44e9999a54a081d436511, State: Initialized, Role: FOLLOWER
I20260810 07:19:04.093966 13411 ts_tablet_manager.cc:1428] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 07:19:04.094094 13410 consensus_queue.cc:260] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.094195 13412 consensus_queue.cc:260] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.094799 13381 heartbeater.cc:503] Master 127.12.117.190:37083 was elected leader, sending a full tablet report...
I20260810 07:19:04.095047 13412 ts_tablet_manager.cc:1428] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 07:19:04.095047 13410 ts_tablet_manager.cc:1428] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260810 07:19:04.095351 13149 heartbeater.cc:503] Master 127.12.117.190:37083 was elected leader, sending a full tablet report...
W20260810 07:19:04.096871 13151 tablet.cc:2369] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:04.142565 13420 raft_consensus.cc:482] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:04.142750 13420 raft_consensus.cc:504] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.144340 13420 leader_election.cc:288] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b17bec14bc8e465283b7fa6412bb9fe3 (127.12.117.130:39607), d197569ddb4c441a9b1d51bb5365b765 (127.12.117.129:46339)
I20260810 07:19:04.147541 13419 raft_consensus.cc:482] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:04.147720 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.144170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:39607 (local address 127.12.117.131:38405)
0810 07:19:04.144326 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.144342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.144428 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.144723 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.144725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.144746 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.144884 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.144892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.145701 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.145704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.146587 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.146597 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.147295 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.147299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.147398 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.147425 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.147493 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.147566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":20,"spinlock_wait_cycles":20864}
I20260810 07:19:04.147740 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.144248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:38405 (local address 127.12.117.130:39607)
0810 07:19:04.144370 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.144374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.144390 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:19:04.144565 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.144569 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.144623 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.144700 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.145001 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.145593 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.146706 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.147165 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.147256 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.147335 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.147567 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.147643 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:04.147840 13419 raft_consensus.cc:504] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.147936 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.144281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:60633 (local address 127.12.117.129:46339)
0810 07:19:04.144495 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.144499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.144778 (+ 279us) server_negotiation.cc:407] Connection header received
0810 07:19:04.144900 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.144903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.144954 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.145022 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.145277 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.145795 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.146910 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.147352 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.147457 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.147503 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.147737 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.147820 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 07:19:04.148285 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.144181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46339 (local address 127.12.117.131:60633)
0810 07:19:04.144687 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.144700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.144808 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.145038 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.145040 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.145057 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.145172 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.145179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.145901 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.145904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.146792 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.146803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.147483 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.147487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.147590 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.147615 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.148058 (+ 443us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.148144 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":152,"spinlock_wait_cycles":896,"thread_start_us":124,"threads_started":1}
I20260810 07:19:04.148345 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "7d025eb828c44e9999a54a081d436511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" is_pre_election: true
I20260810 07:19:04.148562 13069 raft_consensus.cc:2454] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d025eb828c44e9999a54a081d436511 in term 0.
I20260810 07:19:04.148942 12862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "7d025eb828c44e9999a54a081d436511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d197569ddb4c441a9b1d51bb5365b765" is_pre_election: true
I20260810 07:19:04.149096 13177 leader_election.cc:302] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d025eb828c44e9999a54a081d436511, b17bec14bc8e465283b7fa6412bb9fe3; no voters:
I20260810 07:19:04.149261 12862 raft_consensus.cc:2454] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d025eb828c44e9999a54a081d436511 in term 0.
I20260810 07:19:04.149505 13420 raft_consensus.cc:2781] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:04.149597 13420 raft_consensus.cc:482] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:04.149644 13420 raft_consensus.cc:3037] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:04.149681 13419 leader_election.cc:288] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d025eb828c44e9999a54a081d436511 (127.12.117.131:46519), d197569ddb4c441a9b1d51bb5365b765 (127.12.117.129:46339)
I20260810 07:19:04.150939 13420 raft_consensus.cc:504] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.151331 13420 leader_election.cc:288] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [CANDIDATE]: Term 1 election: Requested vote from peers b17bec14bc8e465283b7fa6412bb9fe3 (127.12.117.130:39607), d197569ddb4c441a9b1d51bb5365b765 (127.12.117.129:46339)
I20260810 07:19:04.151630 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "7d025eb828c44e9999a54a081d436511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17bec14bc8e465283b7fa6412bb9fe3"
I20260810 07:19:04.151739 13069 raft_consensus.cc:3037] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:04.152316 12862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "7d025eb828c44e9999a54a081d436511" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d197569ddb4c441a9b1d51bb5365b765"
I20260810 07:19:04.152424 12862 raft_consensus.cc:3037] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:04.152943 13069 raft_consensus.cc:2454] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d025eb828c44e9999a54a081d436511 in term 1.
I20260810 07:19:04.153291 13177 leader_election.cc:302] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d025eb828c44e9999a54a081d436511, b17bec14bc8e465283b7fa6412bb9fe3; no voters:
I20260810 07:19:04.153357 13152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.149256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:46519 (local address 127.12.117.130:52227)
0810 07:19:04.149594 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.149609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.149690 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.150202 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.150204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.150220 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.150359 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.150366 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.151232 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.151235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.152131 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.152141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.152973 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.152976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.153072 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.153095 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.153161 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.153232 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":8,"spinlock_wait_cycles":22400}
I20260810 07:19:04.153542 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.149389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:50933 (local address 127.12.117.129:46339)
0810 07:19:04.149483 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.149488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.149836 (+ 348us) server_negotiation.cc:407] Connection header received
0810 07:19:04.150005 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.150008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.150049 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.150102 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.150530 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.150974 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.152330 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.152867 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.152956 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.153002 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.153346 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.153435 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260810 07:19:04.153671 12862 raft_consensus.cc:2454] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d025eb828c44e9999a54a081d436511 in term 1.
I20260810 07:19:04.153821 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.149849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:52227 (local address 127.12.117.131:46519)
0810 07:19:04.149945 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.149949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.149961 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:04.149999 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.150002 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.150040 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.150114 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:04.150474 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.151107 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.152250 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.152768 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.152872 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.152913 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.153652 (+ 739us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.153723 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:19:04.153891 13425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.149256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46339 (local address 127.12.117.130:50933)
0810 07:19:04.149752 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.149765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:04.149899 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:04.150229 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:04.150231 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:04.150259 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:04.150393 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.150398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.151100 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.151103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.152194 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:04.152203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.153105 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.153109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:04.153212 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.153234 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 07:19:04.153706 (+ 472us) client_negotiation.cc:240] Negotiation successful
0810 07:19:04.153772 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":197,"thread_start_us":121,"threads_started":1}
I20260810 07:19:04.154064 13420 raft_consensus.cc:2781] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:04.154287 13420 raft_consensus.cc:686] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 1 LEADER]: Becoming Leader. State: Replica: 7d025eb828c44e9999a54a081d436511, State: Running, Role: LEADER
I20260810 07:19:04.154282 12862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d197569ddb4c441a9b1d51bb5365b765" is_pre_election: true
I20260810 07:19:04.154341 13308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e25119ae807427e82c8ef0365745506" candidate_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d025eb828c44e9999a54a081d436511" is_pre_election: true
I20260810 07:19:04.154438 12862 raft_consensus.cc:2379] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b17bec14bc8e465283b7fa6412bb9fe3 in current term 1: Already voted for candidate 7d025eb828c44e9999a54a081d436511 in this term.
I20260810 07:19:04.154606 13420 consensus_queue.cc:237] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } }
I20260810 07:19:04.155016 12958 leader_election.cc:302] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b17bec14bc8e465283b7fa6412bb9fe3; no voters: 7d025eb828c44e9999a54a081d436511, d197569ddb4c441a9b1d51bb5365b765
I20260810 07:19:04.155310 13419 raft_consensus.cc:2726] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 07:19:04.157747 12609 catalog_manager.cc:5495] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d025eb828c44e9999a54a081d436511 (127.12.117.131). New cstate: current_term: 1 leader_uuid: "7d025eb828c44e9999a54a081d436511" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d025eb828c44e9999a54a081d436511" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:04.170575 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.167669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46250 (local address 127.12.117.131:46519)
0810 07:19:04.167767 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:04.167770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:04.167783 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:04.167880 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:04.167882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:04.167927 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:04.168004 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:04.168171 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.168625 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.169390 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:04.169595 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:04.169659 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:04.169884 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:04.170248 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:04.170348 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:04.170424 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 07:19:04.170450 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260810 07:19:04.171736 12925 tablet.cc:2369] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:04.175886 12862 raft_consensus.cc:1261] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 1 FOLLOWER]: Refusing update from remote peer 7d025eb828c44e9999a54a081d436511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:04.175926 13069 raft_consensus.cc:1261] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 1 FOLLOWER]: Refusing update from remote peer 7d025eb828c44e9999a54a081d436511: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:04.176476 13426 consensus_queue.cc:1035] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d197569ddb4c441a9b1d51bb5365b765" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:04.176636 13420 consensus_queue.cc:1035] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17bec14bc8e465283b7fa6412bb9fe3" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 39607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:04.184046 13433 mvcc.cc:205] Tried to move back new op lower bound from 7316874625738199040 to 7316874625660272640. Current Snapshot: MvccSnapshot[applied={T|T < 7316874625738199040}]
I20260810 07:19:04.191545 12608 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:04.191682 12608 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:04.193779 12608 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59930fadaf8548959e7fec59cef94506: Sending DeleteTablet for 3 replicas of tablet 6e25119ae807427e82c8ef0365745506
I20260810 07:19:04.194469 13275 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e25119ae807427e82c8ef0365745506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:04 UTC) from {username='slave'} at 127.0.0.1:46234
I20260810 07:19:04.194470 13032 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e25119ae807427e82c8ef0365745506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:04 UTC) from {username='slave'} at 127.0.0.1:38776
I20260810 07:19:04.194672 12833 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e25119ae807427e82c8ef0365745506 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:04 UTC) from {username='slave'} at 127.0.0.1:53958
I20260810 07:19:04.194922 13447 tablet_replica.cc:330] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: stopping tablet replica
I20260810 07:19:04.195106 13450 tablet_replica.cc:330] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765: stopping tablet replica
I20260810 07:19:04.195292 13447 raft_consensus.cc:2229] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:04.195406 13450 raft_consensus.cc:2229] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:04.195720 13447 raft_consensus.cc:2258] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:04.195711 13450 raft_consensus.cc:2258] T 6e25119ae807427e82c8ef0365745506 P d197569ddb4c441a9b1d51bb5365b765 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:04.195748 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12722
I20260810 07:19:04.205896 13447 ts_tablet_manager.cc:1905] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:04.208359 13447 ts_tablet_manager.cc:1918] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:04.208453 13447 log.cc:1196] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/6e25119ae807427e82c8ef0365745506
I20260810 07:19:04.208751 13448 tablet_replica.cc:330] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: stopping tablet replica
I20260810 07:19:04.209064 13448 raft_consensus.cc:2229] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:04.209369 13448 raft_consensus.cc:2258] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:04.209928 13447 ts_tablet_manager.cc:1939] T 6e25119ae807427e82c8ef0365745506 P 7d025eb828c44e9999a54a081d436511: Deleting consensus metadata
I20260810 07:19:04.211443 13448 ts_tablet_manager.cc:1905] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:04.211628 12587 catalog_manager.cc:4841] TS 7d025eb828c44e9999a54a081d436511 (127.12.117.131:46519): tablet 6e25119ae807427e82c8ef0365745506 (table test-table [id=2686ae617f034e3585066177adc036f1]) successfully deleted
I20260810 07:19:04.214074 13448 ts_tablet_manager.cc:1918] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:04.214167 13448 log.cc:1196] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/6e25119ae807427e82c8ef0365745506
I20260810 07:19:04.214438 13448 ts_tablet_manager.cc:1939] T 6e25119ae807427e82c8ef0365745506 P b17bec14bc8e465283b7fa6412bb9fe3: Deleting consensus metadata
I20260810 07:19:04.215627 12590 catalog_manager.cc:4841] TS b17bec14bc8e465283b7fa6412bb9fe3 (127.12.117.130:39607): tablet 6e25119ae807427e82c8ef0365745506 (table test-table [id=2686ae617f034e3585066177adc036f1]) successfully deleted
W20260810 07:19:04.218734 12591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:46339 (error 108)
I20260810 07:19:04.219523 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12927
I20260810 07:19:04.219733 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:04.219440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46339 (local address 127.0.0.1:53966)
0810 07:19:04.219580 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:04.219650 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 07:19:04.227631 12591 catalog_manager.cc:4584] TS d197569ddb4c441a9b1d51bb5365b765 (127.12.117.129:46339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e25119ae807427e82c8ef0365745506: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46339: connect: Connection refused (error 111)
I20260810 07:19:04.249814 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13153
I20260810 07:19:04.265542 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 12564
2026-08-10T07:19:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1087 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:04 dist-test-slave-gjrz krb5kdc[13460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:04 dist-test-slave-gjrz krb5kdc[13460](info): set up 2 sockets
Aug 10 07:19:04 dist-test-slave-gjrz krb5kdc[13460](info): commencing operation
krb5kdc: starting...
W20260810 07:19:05.426298 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:05 dist-test-slave-gjrz krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:05.545181 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36687
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:36687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:05.649420 13491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:05.649652 13491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:05.649689 13491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:05.653506 13491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:05.653551 13491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:05.653569 13491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:05.653589 13491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:05.653604 13491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:05.658488 13491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:36687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:05.659348 13491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:05.660198 13491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:05.667832 13502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:05.667989 13491 server_base.cc:1034] running on GCE node
W20260810 07:19:05.667858 13507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:05.667840 13504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:05.668587 13491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:05.669543 13491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:05.670692 13491 hybrid_clock.cc:648] HybridClock initialized: now 1786346345670673 us; error 34 us; skew 500 ppm
Aug 10 07:19:05 dist-test-slave-gjrz krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346345, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:05.673338 13491 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:05.674137 13491 webserver.cc:460] Webserver started at http://127.12.117.190:44657/ using document root <none> and password file <none>
I20260810 07:19:05.674628 13491 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:05.674693 13491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:05.674867 13491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:05.676770 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "27771046f07c40c48c7201a259800f8a"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "66c5e60bd67b630149522140f6eafcf3"
server_key_iv: "2d2298a9d267c5510f4f29a75fe87746"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.677278 13491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "27771046f07c40c48c7201a259800f8a"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "66c5e60bd67b630149522140f6eafcf3"
server_key_iv: "2d2298a9d267c5510f4f29a75fe87746"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.680686 13491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:05.682852 13519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:05.683945 13491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:05.684054 13491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "27771046f07c40c48c7201a259800f8a"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "66c5e60bd67b630149522140f6eafcf3"
server_key_iv: "2d2298a9d267c5510f4f29a75fe87746"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.684147 13491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:05.698962 13491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:05.701399 13491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:05.701556 13491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:05.707886 13491 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:36687
I20260810 07:19:05.707901 13593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:36687 every 8 connection(s)
I20260810 07:19:05.708732 13491 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:05.710054 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13491
I20260810 07:19:05.710193 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:05.710495 12758 external_mini_cluster.cc:1428] Setting key 4cefcc21fc51492b63780b6adcc0d6d9
I20260810 07:19:05.711776 13594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:05.717660 13594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: Bootstrap starting.
Aug 10 07:19:05 dist-test-slave-gjrz krb5kdc[13460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:05.720026 13594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:05.720685 13594 log.cc:824] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:05.722628 13594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: No bootstrap required, opened a new log
I20260810 07:19:05.723536 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:05.711750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.12.117.190:36687)
0810 07:19:05.712131 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:05.712146 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:05.712181 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:05.712772 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:05.712788 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:05.713089 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:05.713342 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:05.714197 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.715163 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.716301 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.716737 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.716851 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:05.719476 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:05.719480 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:05.719493 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:05.719518 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:05.721034 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:05.721438 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:05.721443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:05.721447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:05.721552 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:05.721823 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:05.721825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:05.721826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:05.722187 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:05.722318 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:05.722604 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 07:19:05.722843 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":112,"threads_started":1}
I20260810 07:19:05.725714 13594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } }
I20260810 07:19:05.725965 13594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:05.726035 13594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27771046f07c40c48c7201a259800f8a, State: Initialized, Role: FOLLOWER
I20260810 07:19:05.726390 13594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } }
I20260810 07:19:05.726500 13594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:05.726552 13594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:05.726619 13594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:05.727393 13594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } }
I20260810 07:19:05.727681 13594 leader_election.cc:302] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27771046f07c40c48c7201a259800f8a; no voters:
I20260810 07:19:05.727911 13594 leader_election.cc:288] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:05.728039 13603 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:05.728256 13603 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [term 1 LEADER]: Becoming Leader. State: Replica: 27771046f07c40c48c7201a259800f8a, State: Running, Role: LEADER
I20260810 07:19:05.728528 13603 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } }
I20260810 07:19:05.728983 13594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:05.730370 13606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27771046f07c40c48c7201a259800f8a. Latest consensus state: current_term: 1 leader_uuid: "27771046f07c40c48c7201a259800f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } } }
I20260810 07:19:05.730392 13605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27771046f07c40c48c7201a259800f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27771046f07c40c48c7201a259800f8a" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36687 } } }
I20260810 07:19:05.730491 13606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:05.730497 13605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:05.733827 13630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:05.733891 13630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:05.734006 13620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:05.734925 13620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:05.739831 13620 catalog_manager.cc:1347] Generated new cluster ID: 507f2fbe9fc444069e248527271b3910
I20260810 07:19:05.739895 13620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:05.754494 13620 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:05.755323 13620 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:05.761530 13620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: Generated new TSK 0
I20260810 07:19:05.762215 13620 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.117.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
Entry for principal kudu/127.12.117.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab.
I20260810 07:19:05.792191 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:05.899283 13636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:05.899505 13636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:05.899552 13636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:05.903378 13636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:05.903420 13636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:05.903472 13636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:19:05.908390 13636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.129.keytab
--principal=kudu/127.12.117.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:05.909219 13636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:05.910090 13636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:05.918756 13652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:05.918779 13649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:05.918773 13646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:05.918983 13636 server_base.cc:1034] running on GCE node
I20260810 07:19:05.919447 13636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:05.920018 13636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:05.921175 13636 hybrid_clock.cc:648] HybridClock initialized: now 1786346345921159 us; error 26 us; skew 500 ppm
Aug 10 07:19:05 dist-test-slave-gjrz krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346345, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:05.923714 13636 init.cc:377] Logged in from keytab as kudu/127.12.117.129@KRBTEST.COM (short username kudu)
I20260810 07:19:05.924466 13636 webserver.cc:460] Webserver started at http://127.12.117.129:37647/ using document root <none> and password file <none>
I20260810 07:19:05.924968 13636 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:05.925031 13636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:05.925220 13636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:05.927129 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "94a6960f60aa4c8fa7406c672c12002d"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "69c071d1121a1ae36d9ff4c762d1398c"
server_key_iv: "668380e4c9f90895fb150e81f8cd91ab"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.927742 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "94a6960f60aa4c8fa7406c672c12002d"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "69c071d1121a1ae36d9ff4c762d1398c"
server_key_iv: "668380e4c9f90895fb150e81f8cd91ab"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.931885 13636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 07:19:05.934142 13662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:05.935310 13636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:05.935421 13636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "94a6960f60aa4c8fa7406c672c12002d"
format_stamp: "Formatted at 2026-08-10 07:19:05 on dist-test-slave-gjrz"
server_key: "69c071d1121a1ae36d9ff4c762d1398c"
server_key_iv: "668380e4c9f90895fb150e81f8cd91ab"
server_key_version: "encryptionkey@0"
I20260810 07:19:05.935513 13636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:05.958191 13636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:05.960682 13636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:05.960839 13636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:05.961344 13636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:05.962103 13636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:05.962150 13636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:05.962198 13636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:05.962229 13636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:05.970515 13636 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:46411
I20260810 07:19:05.970569 13845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:46411 every 8 connection(s)
I20260810 07:19:05.971378 13636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:05.977523 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13636
I20260810 07:19:05.977627 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:05.977890 12758 external_mini_cluster.cc:1428] Setting key 43ea5bfb383030c947b5deed48fb13a6
Aug 10 07:19:05 dist-test-slave-gjrz krb5kdc[13460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346345, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.129@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:05.984308 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:05.973036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:37747 (local address 127.12.117.190:36687)
0810 07:19:05.973170 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:05.973174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:05.973981 (+ 807us) server_negotiation.cc:407] Connection header received
0810 07:19:05.975182 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:05.975184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:05.975231 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:05.975345 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:05.976401 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.976924 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.978084 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.978362 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.978444 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:05.980756 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:05.980760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:05.980763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:05.980793 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:05.982222 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:05.982785 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:05.982788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:05.982789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:05.982874 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:05.983214 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:05.983216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:05.983217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:05.983415 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:05.983491 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:05.984057 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 07:19:05.984182 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:05.985317 13851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:05.973406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:36687 (local address 127.12.117.129:37747)
0810 07:19:05.973839 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:05.973880 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:05.974997 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:05.975484 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:05.975490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:05.975817 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:05.976283 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:05.976296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.977013 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.977015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:05.977962 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:05.977971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:05.978490 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:05.978495 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:05.978664 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:05.979197 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:05.979213 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:05.980604 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:05.982337 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:05.982343 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:05.982356 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:05.982676 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:05.982958 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:05.982961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:05.982963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:05.983133 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:05.983507 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:05.983513 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:05.983813 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 07:19:05.984038 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 07:19:05.984355 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":122,"threads_started":1}
I20260810 07:19:05.986413 13846 heartbeater.cc:347] Connected to a master server at 127.12.117.190:36687
I20260810 07:19:05.986658 13846 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:05.987191 13846 heartbeater.cc:511] Master 127.12.117.190:36687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.117.130@KRBTEST.COM; defaulting to no policy
I20260810 07:19:05.988716 13538 ts_manager.cc:194] Registered new tserver with Master: 94a6960f60aa4c8fa7406c672c12002d (127.12.117.129:46411)
I20260810 07:19:05.989818 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.129@KRBTEST.COM'} at 127.12.117.129:37747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
Entry for principal kudu/127.12.117.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab.
I20260810 07:19:06.004063 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:06.112205 13855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:06.112425 13855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:06.112463 13855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:06.116299 13855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:06.116346 13855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:06.116400 13855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:19:06.121371 13855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.130.keytab
--principal=kudu/127.12.117.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:06.122248 13855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:06.123134 13855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:06.132562 13860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.132562 13865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.132664 13862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.132712 13855 server_base.cc:1034] running on GCE node
I20260810 07:19:06.133247 13855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:06.133838 13855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:06.134999 13855 hybrid_clock.cc:648] HybridClock initialized: now 1786346346134974 us; error 33 us; skew 500 ppm
Aug 10 07:19:06 dist-test-slave-gjrz krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:06.137594 13855 init.cc:377] Logged in from keytab as kudu/127.12.117.130@KRBTEST.COM (short username kudu)
I20260810 07:19:06.138445 13855 webserver.cc:460] Webserver started at http://127.12.117.130:39971/ using document root <none> and password file <none>
I20260810 07:19:06.138965 13855 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:06.139029 13855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:06.139240 13855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:06.141201 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "bf28d900ca03924a118ed3da9dfd28b9"
server_key_iv: "13d03ea80483dbf5ee1768cb7cef6f49"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.141738 13855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "bf28d900ca03924a118ed3da9dfd28b9"
server_key_iv: "13d03ea80483dbf5ee1768cb7cef6f49"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.145222 13855 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:06.147521 13880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.148641 13855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:06.148742 13855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "bf28d900ca03924a118ed3da9dfd28b9"
server_key_iv: "13d03ea80483dbf5ee1768cb7cef6f49"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.148835 13855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:06.161000 13855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:06.163431 13855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:06.163594 13855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:06.164114 13855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:06.164880 13855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:06.164923 13855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.164963 13855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:06.164992 13855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.173137 13855 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:40413
I20260810 07:19:06.173166 14049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:40413 every 8 connection(s)
I20260810 07:19:06.173954 13855 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:06.179487 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 13855
I20260810 07:19:06.179610 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:06.179940 12758 external_mini_cluster.cc:1428] Setting key 9502f32ae029b8603ba4f9f0b7d70293
Aug 10 07:19:06 dist-test-slave-gjrz krb5kdc[13460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.130@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:06.186959 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.175625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:41771 (local address 127.12.117.190:36687)
0810 07:19:06.175753 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.175756 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.176564 (+ 808us) server_negotiation.cc:407] Connection header received
0810 07:19:06.177755 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.177758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.177806 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.177877 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.178919 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.179494 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.180917 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.181183 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.181267 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.183612 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.183616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.183618 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:06.183642 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.184938 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.185473 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.185476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.185478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.185553 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.185912 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.185915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.185916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.186100 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.186175 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.186737 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.186842 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:06.188043 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.175976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:36687 (local address 127.12.117.130:41771)
0810 07:19:06.176422 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.176462 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.177574 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.178023 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.178030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.178358 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:06.178810 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.178823 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.179635 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.179639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.180805 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.180816 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.181325 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.181329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.181476 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.181995 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:06.182010 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:06.183456 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:06.185050 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:06.185056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:06.185069 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:06.185372 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:06.185645 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:06.185648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:06.185650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:06.185822 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:06.186198 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:06.186204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:06.186496 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.186714 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.187054 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260810 07:19:06.189133 14050 heartbeater.cc:347] Connected to a master server at 127.12.117.190:36687
I20260810 07:19:06.189343 14050 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:06.189781 14050 heartbeater.cc:511] Master 127.12.117.190:36687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.117.131@KRBTEST.COM; defaulting to no policy
I20260810 07:19:06.190754 13538 ts_manager.cc:194] Registered new tserver with Master: 5a5f2c6fdf8441ce9b4ff3e7c56be24b (127.12.117.130:40413)
I20260810 07:19:06.191321 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.130@KRBTEST.COM'} at 127.12.117.130:41771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
Entry for principal kudu/127.12.117.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab.
I20260810 07:19:06.205533 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:06.313992 14059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:06.314219 14059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:06.314260 14059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:06.318240 14059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:06.318290 14059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:06.318356 14059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:19:06.323369 14059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.131.keytab
--principal=kudu/127.12.117.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:36687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:06.324270 14059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:06.325199 14059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:06.334659 14075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.334652 14070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.334760 14059 server_base.cc:1034] running on GCE node
W20260810 07:19:06.334652 14072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.335445 14059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:06.336028 14059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:06.337184 14059 hybrid_clock.cc:648] HybridClock initialized: now 1786346346337160 us; error 40 us; skew 500 ppm
Aug 10 07:19:06 dist-test-slave-gjrz krb5kdc[13460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:06.339856 14059 init.cc:377] Logged in from keytab as kudu/127.12.117.131@KRBTEST.COM (short username kudu)
I20260810 07:19:06.340728 14059 webserver.cc:460] Webserver started at http://127.12.117.131:39133/ using document root <none> and password file <none>
I20260810 07:19:06.341255 14059 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:06.341320 14059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:06.341507 14059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:06.343518 14059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "7bf7aef026bf480fa5f4e963e08fb093"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "9d830499a03d7c138025fa9573ee9b6d"
server_key_iv: "0ea12a06daa108808da0cdfd3f77dc20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.344049 14059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "7bf7aef026bf480fa5f4e963e08fb093"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "9d830499a03d7c138025fa9573ee9b6d"
server_key_iv: "0ea12a06daa108808da0cdfd3f77dc20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.347527 14059 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:06.349752 14084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.350921 14059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:06.351024 14059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "7bf7aef026bf480fa5f4e963e08fb093"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "9d830499a03d7c138025fa9573ee9b6d"
server_key_iv: "0ea12a06daa108808da0cdfd3f77dc20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.351120 14059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:06.374764 14059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:06.377557 14059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:06.377729 14059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:06.378276 14059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:06.379076 14059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:06.379122 14059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.379172 14059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:06.379202 14059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.387256 14059 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:40861
I20260810 07:19:06.387343 14276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:40861 every 8 connection(s)
I20260810 07:19:06.388119 14059 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:06.391403 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14059
I20260810 07:19:06.391515 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:06.391785 12758 external_mini_cluster.cc:1428] Setting key b7a92eb38a175639aa0fd0bf59c4b147
Aug 10 07:19:06 dist-test-slave-gjrz krb5kdc[13460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346346, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.131@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:06.400549 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.389722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:55163 (local address 127.12.117.190:36687)
0810 07:19:06.389844 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.389848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.390613 (+ 765us) server_negotiation.cc:407] Connection header received
0810 07:19:06.391931 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.391935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.391992 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.392096 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.393438 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.393994 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.395066 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.395300 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.395375 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.397557 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.397561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.397563 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:06.397588 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.398793 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.399226 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.399229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.399230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.399349 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.399628 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.399630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.399632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.399801 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.399905 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.400327 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.400434 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:06.401234 14279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.390064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:36687 (local address 127.12.117.131:55163)
0810 07:19:06.390476 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.390514 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.391719 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.392265 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.392275 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.392729 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:06.393313 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.393334 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.394107 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.394110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.394970 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.394979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.395419 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.395422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.395554 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.396066 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:06.396081 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:06.397401 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:06.398900 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:06.398905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:06.398915 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:06.399149 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:06.399428 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:06.399430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:06.399431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:06.399551 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:06.399873 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:06.399877 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:06.400103 (+ 226us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.400254 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.400476 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":112,"threads_started":1}
I20260810 07:19:06.402274 14277 heartbeater.cc:347] Connected to a master server at 127.12.117.190:36687
I20260810 07:19:06.402472 14277 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:06.402894 14277 heartbeater.cc:511] Master 127.12.117.190:36687 requested a full tablet report, sending...
I20260810 07:19:06.403812 13538 ts_manager.cc:194] Registered new tserver with Master: 7bf7aef026bf480fa5f4e963e08fb093 (127.12.117.131:40861)
I20260810 07:19:06.404392 13538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.117.131@KRBTEST.COM'} at 127.12.117.131:55163
I20260810 07:19:06.405439 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:06.413679 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.407384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.12.117.190:36687)
0810 07:19:06.407486 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.407490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.407644 (+ 154us) server_negotiation.cc:407] Connection header received
0810 07:19:06.407764 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.407767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.407806 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.407864 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.408487 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.408965 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.409749 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.409958 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.410038 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.411140 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.411144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.411147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:06.411171 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.412203 (+ 1032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.412565 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.412568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.412570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.412655 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:06.412890 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:06.412893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:06.412895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:06.413102 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.413213 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.413431 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.413531 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:19:06.415931 13538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:06.418725 13538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:06.431931 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46411 (local address 127.0.0.1:37304)
0810 07:19:06.426370 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.426415 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.426523 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.427679 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.427684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.427705 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:06.427818 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.427826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.429131 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.429134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.430414 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.430420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431437 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.431547 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431593 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.431693 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.431805 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":30,"thread_start_us":55,"threads_started":1}
I20260810 07:19:06.431939 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.12.117.129:46411)
0810 07:19:06.427189 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.427201 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.427225 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:06.427325 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.427330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.427530 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.427678 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:06.427946 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.429007 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.430523 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431323 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431438 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431612 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.431717 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.431805 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":146,"threads_started":1}
I20260810 07:19:06.432231 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.12.117.130:40413)
0810 07:19:06.427255 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.427285 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.427309 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:06.427570 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.427602 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.427744 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.427893 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:06.428315 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.429348 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.430687 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431470 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431567 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431663 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.432000 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.432104 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":112,"threads_started":1}
I20260810 07:19:06.432423 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54614 (local address 127.12.117.131:40861)
0810 07:19:06.427474 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.427486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.427505 (+ 19us) server_negotiation.cc:407] Connection header received
0810 07:19:06.427565 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.427570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.427740 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.427893 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:06.428097 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.428902 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.430556 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431379 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431493 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431627 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.432217 (+ 590us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.432305 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":110,"threads_started":1}
I20260810 07:19:06.432128 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40861 (local address 127.0.0.1:54614)
0810 07:19:06.427229 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.427242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.427360 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.427863 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.427866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.427883 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:06.427978 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.427985 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.429015 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.429018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.430404 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.430416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431693 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.431799 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431854 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.431923 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.432003 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":26,"thread_start_us":89,"threads_started":1}
I20260810 07:19:06.432572 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.426495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:40413 (local address 127.0.0.1:49598)
0810 07:19:06.426920 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.426933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.427425 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.428057 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.428060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.428079 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:06.428183 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.428189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.429494 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.429497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.430557 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.430567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.431713 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.431716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.431803 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.431868 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.431923 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.432458 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":477,"thread_start_us":105,"threads_started":1}
I20260810 07:19:06.434923 13745 tablet_service.cc:1467] Processing CreateTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 (DEFAULT_TABLE table=test-table [id=84b84c1dff374b3085b3eb815217fd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:06.435209 14169 tablet_service.cc:1467] Processing CreateTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 (DEFAULT_TABLE table=test-table [id=84b84c1dff374b3085b3eb815217fd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:06.435211 13948 tablet_service.cc:1467] Processing CreateTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 (DEFAULT_TABLE table=test-table [id=84b84c1dff374b3085b3eb815217fd4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:06.435871 13745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e0b474ebc14f50ba6d5c0379cb4789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:06.436229 14169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e0b474ebc14f50ba6d5c0379cb4789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:06.436229 13948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1e0b474ebc14f50ba6d5c0379cb4789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:06.442512 14304 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Bootstrap starting.
I20260810 07:19:06.443732 14305 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Bootstrap starting.
I20260810 07:19:06.443732 14306 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Bootstrap starting.
I20260810 07:19:06.444607 14304 tablet_bootstrap.cc:654] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:06.445600 14304 log.cc:824] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:06.445854 14305 tablet_bootstrap.cc:654] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:06.445854 14306 tablet_bootstrap.cc:654] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:06.447100 14305 log.cc:824] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:06.447101 14306 log.cc:824] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:06.447458 14304 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: No bootstrap required, opened a new log
I20260810 07:19:06.447659 14304 ts_tablet_manager.cc:1397] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:06.450412 14304 raft_consensus.cc:348] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.450628 14304 raft_consensus.cc:374] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:06.450700 14304 raft_consensus.cc:729] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94a6960f60aa4c8fa7406c672c12002d, State: Initialized, Role: FOLLOWER
I20260810 07:19:06.451181 14304 consensus_queue.cc:260] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.451949 13846 heartbeater.cc:503] Master 127.12.117.190:36687 was elected leader, sending a full tablet report...
I20260810 07:19:06.452225 14304 ts_tablet_manager.cc:1428] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260810 07:19:06.452430 14305 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: No bootstrap required, opened a new log
I20260810 07:19:06.452461 14306 tablet_bootstrap.cc:492] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: No bootstrap required, opened a new log
I20260810 07:19:06.452677 14305 ts_tablet_manager.cc:1397] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 07:19:06.452716 14306 ts_tablet_manager.cc:1397] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 07:19:06.456161 14305 raft_consensus.cc:348] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.456357 14305 raft_consensus.cc:374] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:06.456439 14305 raft_consensus.cc:729] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf7aef026bf480fa5f4e963e08fb093, State: Initialized, Role: FOLLOWER
I20260810 07:19:06.456352 14306 raft_consensus.cc:348] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.456615 14306 raft_consensus.cc:374] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:06.456673 14306 raft_consensus.cc:729] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a5f2c6fdf8441ce9b4ff3e7c56be24b, State: Initialized, Role: FOLLOWER
I20260810 07:19:06.456905 14305 consensus_queue.cc:260] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.457021 14306 consensus_queue.cc:260] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.457481 14277 heartbeater.cc:503] Master 127.12.117.190:36687 was elected leader, sending a full tablet report...
I20260810 07:19:06.457603 14050 heartbeater.cc:503] Master 127.12.117.190:36687 was elected leader, sending a full tablet report...
I20260810 07:19:06.457798 14305 ts_tablet_manager.cc:1428] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:06.457903 14306 ts_tablet_manager.cc:1428] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:06.459484 14315 raft_consensus.cc:482] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:06.459662 14315 raft_consensus.cc:504] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.460777 14315 leader_election.cc:288] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94a6960f60aa4c8fa7406c672c12002d (127.12.117.129:46411), 7bf7aef026bf480fa5f4e963e08fb093 (127.12.117.131:40861)
I20260810 07:19:06.464533 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.460981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:47847 (local address 127.12.117.129:46411)
0810 07:19:06.461406 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.461409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.461420 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:06.461455 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.461457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.461491 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.461555 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:06.461805 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.462334 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.463616 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.464046 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.464152 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.464203 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.464358 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.464425 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360}
I20260810 07:19:06.464534 14053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.460914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46411 (local address 127.12.117.130:47847)
0810 07:19:06.461013 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.461028 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.461104 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.461591 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.461594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.461612 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:06.461712 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.461719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.462457 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.462461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.463498 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.463508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.464152 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.464156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.464256 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.464273 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.464336 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.464410 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 07:19:06.465003 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.461752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:39423 (local address 127.12.117.131:40861)
0810 07:19:06.461862 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.461876 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.461887 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:06.461920 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.461922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.461953 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.462015 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:06.462297 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.463019 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.464133 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.464511 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.464612 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.464659 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.464852 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.464926 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:06.465096 13771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e0b474ebc14f50ba6d5c0379cb4789" candidate_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94a6960f60aa4c8fa7406c672c12002d" is_pre_election: true
I20260810 07:19:06.465294 14316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.461153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:40861 (local address 127.12.117.130:39423)
0810 07:19:06.461394 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.461406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:06.461483 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:06.462041 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:06.462044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:06.462060 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:06.462183 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.462189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.463140 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.463142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.464012 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:06.464022 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.464618 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.464622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:06.464716 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.464740 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 07:19:06.465115 (+ 375us) client_negotiation.cc:240] Negotiation successful
0810 07:19:06.465183 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260810 07:19:06.465373 13771 raft_consensus.cc:2454] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a5f2c6fdf8441ce9b4ff3e7c56be24b in term 0.
I20260810 07:19:06.465785 13882 leader_election.cc:302] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5f2c6fdf8441ce9b4ff3e7c56be24b, 94a6960f60aa4c8fa7406c672c12002d; no voters:
I20260810 07:19:06.465994 14315 raft_consensus.cc:2781] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:06.466092 14315 raft_consensus.cc:482] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:06.466033 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e0b474ebc14f50ba6d5c0379cb4789" candidate_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf7aef026bf480fa5f4e963e08fb093" is_pre_election: true
I20260810 07:19:06.466145 14315 raft_consensus.cc:3037] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:06.466287 14202 raft_consensus.cc:2454] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a5f2c6fdf8441ce9b4ff3e7c56be24b in term 0.
I20260810 07:19:06.467132 14315 raft_consensus.cc:504] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
I20260810 07:19:06.467444 14315 leader_election.cc:288] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [CANDIDATE]: Term 1 election: Requested vote from peers 94a6960f60aa4c8fa7406c672c12002d (127.12.117.129:46411), 7bf7aef026bf480fa5f4e963e08fb093 (127.12.117.131:40861)
I20260810 07:19:06.467751 13771 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e0b474ebc14f50ba6d5c0379cb4789" candidate_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94a6960f60aa4c8fa7406c672c12002d"
I20260810 07:19:06.467796 14202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1e0b474ebc14f50ba6d5c0379cb4789" candidate_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf7aef026bf480fa5f4e963e08fb093"
I20260810 07:19:06.467861 13771 raft_consensus.cc:3037] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:06.467893 14202 raft_consensus.cc:3037] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:06.468801 13771 raft_consensus.cc:2454] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5f2c6fdf8441ce9b4ff3e7c56be24b in term 1.
I20260810 07:19:06.468806 14202 raft_consensus.cc:2454] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a5f2c6fdf8441ce9b4ff3e7c56be24b in term 1.
I20260810 07:19:06.469055 13882 leader_election.cc:302] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a5f2c6fdf8441ce9b4ff3e7c56be24b, 7bf7aef026bf480fa5f4e963e08fb093; no voters:
I20260810 07:19:06.469208 14315 raft_consensus.cc:2781] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:06.469403 14315 raft_consensus.cc:686] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 1 LEADER]: Becoming Leader. State: Replica: 5a5f2c6fdf8441ce9b4ff3e7c56be24b, State: Running, Role: LEADER
I20260810 07:19:06.469666 14315 consensus_queue.cc:237] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } }
W20260810 07:19:06.472383 13849 tablet.cc:2369] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:06.472398 13538 catalog_manager.cc:5495] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a5f2c6fdf8441ce9b4ff3e7c56be24b (127.12.117.130). New cstate: current_term: 1 leader_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a5f2c6fdf8441ce9b4ff3e7c56be24b" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40413 } health_report { overall_health: HEALTHY } } }
I20260810 07:19:06.492985 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.489156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.12.117.130:40413)
0810 07:19:06.489278 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.489282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.489295 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:06.489386 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.489389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.489439 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.489523 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:06.489756 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.490232 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.491070 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.491417 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.491511 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.491921 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:06.492451 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:06.492566 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.492719 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.492803 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:06.499361 14202 raft_consensus.cc:1261] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 1 FOLLOWER]: Refusing update from remote peer 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:06.499532 13771 raft_consensus.cc:1261] T a1e0b474ebc14f50ba6d5c0379cb4789 P 94a6960f60aa4c8fa7406c672c12002d [term 1 FOLLOWER]: Refusing update from remote peer 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:06.500166 14315 consensus_queue.cc:1035] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [LEADER]: Connected to new peer: Peer: permanent_uuid: "94a6960f60aa4c8fa7406c672c12002d" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 46411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:06.500346 14317 consensus_queue.cc:1035] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bf7aef026bf480fa5f4e963e08fb093" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 40861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:06.511006 14323 mvcc.cc:205] Tried to move back new op lower bound from 7316874635254059008 to 7316874635141865472. Current Snapshot: MvccSnapshot[applied={T|T < 7316874635254059008}]
I20260810 07:19:06.513975 14324 mvcc.cc:205] Tried to move back new op lower bound from 7316874635254059008 to 7316874635141865472. Current Snapshot: MvccSnapshot[applied={T|T < 7316874635254059008}]
I20260810 07:19:06.520282 13538 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:06.520440 13538 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:06.522904 13538 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27771046f07c40c48c7201a259800f8a: Sending DeleteTablet for 3 replicas of tablet a1e0b474ebc14f50ba6d5c0379cb4789
I20260810 07:19:06.523712 13745 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:06 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:37304
I20260810 07:19:06.523751 13948 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:06 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:49598
I20260810 07:19:06.524175 14344 tablet_replica.cc:330] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: stopping tablet replica
I20260810 07:19:06.524178 14169 tablet_service.cc:1514] Processing DeleteTablet for tablet a1e0b474ebc14f50ba6d5c0379cb4789 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:06 UTC) from {username='kudu', principal='kudu/127.12.117.190@KRBTEST.COM'} at 127.0.0.1:54614
I20260810 07:19:06.524456 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13636
I20260810 07:19:06.524453 14344 raft_consensus.cc:2229] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 1 LEADER]: Raft consensus shutting down.
I20260810 07:19:06.524749 14344 raft_consensus.cc:2258] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:06.539218 14344 ts_tablet_manager.cc:1905] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:06.542109 14344 ts_tablet_manager.cc:1918] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:06.542218 14344 log.cc:1196] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/a1e0b474ebc14f50ba6d5c0379cb4789
I20260810 07:19:06.542541 14344 ts_tablet_manager.cc:1939] T a1e0b474ebc14f50ba6d5c0379cb4789 P 5a5f2c6fdf8441ce9b4ff3e7c56be24b: Deleting consensus metadata
I20260810 07:19:06.543488 14346 tablet_replica.cc:330] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: stopping tablet replica
I20260810 07:19:06.543749 13521 catalog_manager.cc:4841] TS 5a5f2c6fdf8441ce9b4ff3e7c56be24b (127.12.117.130:40413): tablet a1e0b474ebc14f50ba6d5c0379cb4789 (table test-table [id=84b84c1dff374b3085b3eb815217fd4d]) successfully deleted
I20260810 07:19:06.543972 14346 raft_consensus.cc:2229] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:06.544291 14346 raft_consensus.cc:2258] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:06.545635 14346 ts_tablet_manager.cc:1905] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:06.548230 14346 ts_tablet_manager.cc:1918] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 07:19:06.548223 13522 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:46411 (error 108)
I20260810 07:19:06.548383 14346 log.cc:1196] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/a1e0b474ebc14f50ba6d5c0379cb4789
I20260810 07:19:06.548451 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13855
I20260810 07:19:06.551568 14346 ts_tablet_manager.cc:1939] T a1e0b474ebc14f50ba6d5c0379cb4789 P 7bf7aef026bf480fa5f4e963e08fb093: Deleting consensus metadata
I20260810 07:19:06.554513 13522 catalog_manager.cc:4841] TS 7bf7aef026bf480fa5f4e963e08fb093 (127.12.117.131:40861): tablet a1e0b474ebc14f50ba6d5c0379cb4789 (table test-table [id=84b84c1dff374b3085b3eb815217fd4d]) successfully deleted
I20260810 07:19:06.554659 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.554377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:46411 (local address 127.0.0.1:37308)
0810 07:19:06.554503 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:06.554578 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 07:19:06.554863 13522 catalog_manager.cc:4584] TS 94a6960f60aa4c8fa7406c672c12002d (127.12.117.129:46411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a1e0b474ebc14f50ba6d5c0379cb4789: Network error: Client connection negotiation failed: client connection to 127.12.117.129:46411: connect: Connection refused (error 111)
I20260810 07:19:06.559505 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14059
I20260810 07:19:06.586452 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 13491
2026-08-10T07:19:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2319 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T07:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:06Z Disabled control of system clock
I20260810 07:19:06.620345 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37029
--webserver_interface=127.12.117.190
--webserver_port=0
--builtin_ntp_servers=127.12.117.148:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:19:06.724759 14354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:06.724998 14354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:06.725046 14354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:06.728911 14354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:06.728960 14354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:06.728981 14354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:06.729000 14354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:06.729022 14354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:06.733609 14354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:06.734500 14354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:06.735738 14354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:06.743294 14360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.743328 14365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.743291 14361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.743559 14354 server_base.cc:1034] running on GCE node
I20260810 07:19:06.744115 14354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:06.745126 14354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:06.746282 14354 hybrid_clock.cc:648] HybridClock initialized: now 1786346346746267 us; error 26 us; skew 500 ppm
I20260810 07:19:06.747969 14354 webserver.cc:460] Webserver started at http://127.12.117.190:46029/ using document root <none> and password file <none>
I20260810 07:19:06.748471 14354 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:06.748536 14354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:06.748741 14354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:06.750589 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "3eed6c58ea4844399ddcaf7230c52c22"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "1af611ac3578fc6fbe219145e5c0856b"
server_key_iv: "c448172c6ef69981a29a19b5e8271a30"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.751101 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "3eed6c58ea4844399ddcaf7230c52c22"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "1af611ac3578fc6fbe219145e5c0856b"
server_key_iv: "c448172c6ef69981a29a19b5e8271a30"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.754364 14354 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 07:19:06.756481 14377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.757547 14354 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:06.757660 14354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "3eed6c58ea4844399ddcaf7230c52c22"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "1af611ac3578fc6fbe219145e5c0856b"
server_key_iv: "c448172c6ef69981a29a19b5e8271a30"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.757743 14354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:06.771191 14354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:06.771822 14354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:06.771981 14354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:06.778247 14354 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37029
I20260810 07:19:06.778282 14463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37029 every 8 connection(s)
I20260810 07:19:06.779089 14354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:06.782025 14464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:06.784874 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14354
I20260810 07:19:06.784982 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:06.785246 12758 external_mini_cluster.cc:1428] Setting key 30dc3b861f52d645940bbb6fcfeaaf41
I20260810 07:19:06.787468 14464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: Bootstrap starting.
I20260810 07:19:06.789619 14464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:06.790313 14464 log.cc:824] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:06.791989 14464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: No bootstrap required, opened a new log
I20260810 07:19:06.793413 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:06.786464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.12.117.190:37029)
0810 07:19:06.787018 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:06.787032 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:06.787065 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:06.787710 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:06.787731 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:06.788102 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:06.788356 (+ 254us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:06.788730 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.789642 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.790515 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:06.790968 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:06.791111 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:06.791883 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:06.791888 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:06.791901 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:06.792268 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:06.792326 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:06.792346 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:06.792429 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:06.792564 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 07:19:06.792797 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
I20260810 07:19:06.795579 14464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } }
I20260810 07:19:06.795786 14464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:06.795842 14464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3eed6c58ea4844399ddcaf7230c52c22, State: Initialized, Role: FOLLOWER
I20260810 07:19:06.796234 14464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } }
I20260810 07:19:06.796360 14464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:06.796422 14464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:06.796516 14464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:06.797267 14464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } }
I20260810 07:19:06.797559 14464 leader_election.cc:302] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3eed6c58ea4844399ddcaf7230c52c22; no voters:
I20260810 07:19:06.797812 14464 leader_election.cc:288] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:06.797891 14481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:06.798094 14481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [term 1 LEADER]: Becoming Leader. State: Replica: 3eed6c58ea4844399ddcaf7230c52c22, State: Running, Role: LEADER
I20260810 07:19:06.798373 14481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } }
I20260810 07:19:06.798864 14464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:06.799851 14483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3eed6c58ea4844399ddcaf7230c52c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } } }
I20260810 07:19:06.799868 14485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3eed6c58ea4844399ddcaf7230c52c22. Latest consensus state: current_term: 1 leader_uuid: "3eed6c58ea4844399ddcaf7230c52c22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3eed6c58ea4844399ddcaf7230c52c22" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37029 } } }
I20260810 07:19:06.800071 14483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:06.800101 14485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:06.801144 14493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:06.803864 14493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:06.808813 14493 catalog_manager.cc:1347] Generated new cluster ID: 67589b39f6f64c2e9625d5dfc96bd0a3
I20260810 07:19:06.808871 14493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:06.817479 14493 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:06.818575 14493 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:06.825848 14493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: Generated new TSK 0
I20260810 07:19:06.826413 14493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:06.828685 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.129:0
--local_ip_for_outbound_sockets=127.12.117.129
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--builtin_ntp_servers=127.12.117.148:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 07:19:06.937609 14503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:06.937836 14503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:06.937876 14503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:06.941820 14503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:06.941880 14503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:06.941948 14503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.129
I20260810 07:19:06.946487 14503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.117.129
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.129
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:06.947448 14503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:06.948720 14503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:06.957633 14516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.957716 14513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:06.957647 14511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:06.957821 14503 server_base.cc:1034] running on GCE node
I20260810 07:19:06.958252 14503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:06.958827 14503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:06.959980 14503 hybrid_clock.cc:648] HybridClock initialized: now 1786346346959953 us; error 38 us; skew 500 ppm
I20260810 07:19:06.961607 14503 webserver.cc:460] Webserver started at http://127.12.117.129:43649/ using document root <none> and password file <none>
I20260810 07:19:06.962145 14503 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:06.962208 14503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:06.962406 14503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:06.964351 14503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data/instance:
uuid: "13096522bcd743d7a24d116271f23928"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "d4d89280432d67fad3ef76701c0d61be"
server_key_iv: "3b3544c7d743438e2f6a5a874e578e20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.964885 14503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance:
uuid: "13096522bcd743d7a24d116271f23928"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "d4d89280432d67fad3ef76701c0d61be"
server_key_iv: "3b3544c7d743438e2f6a5a874e578e20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.968349 14503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:06.970623 14524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:06.971823 14503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:06.971930 14503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal
uuid: "13096522bcd743d7a24d116271f23928"
format_stamp: "Formatted at 2026-08-10 07:19:06 on dist-test-slave-gjrz"
server_key: "d4d89280432d67fad3ef76701c0d61be"
server_key_iv: "3b3544c7d743438e2f6a5a874e578e20"
server_key_version: "encryptionkey@0"
I20260810 07:19:06.972025 14503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:07.005149 14503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:07.005795 14503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:07.005961 14503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:07.006524 14503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:07.007668 14503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:07.007728 14503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.007784 14503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:07.007812 14503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.016554 14503 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.129:34307
I20260810 07:19:07.016578 14721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.129:34307 every 8 connection(s)
I20260810 07:19:07.017424 14503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/data/info.pb
I20260810 07:19:07.025550 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14503
I20260810 07:19:07.025666 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-0/wal/instance
I20260810 07:19:07.025936 12758 external_mini_cluster.cc:1428] Setting key fef2b8aa69074dd0f9c55c5a36274b94
I20260810 07:19:07.026387 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.019207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:57895 (local address 127.12.117.190:37029)
0810 07:19:07.019350 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.019354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.020351 (+ 997us) server_negotiation.cc:407] Connection header received
0810 07:19:07.021728 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.021732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.021793 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.021885 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.022780 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.023367 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.024509 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.024745 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.024835 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.025918 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.025922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.025925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.025939 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.025961 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.025969 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.026027 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.026178 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.026241 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 07:19:07.027379 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.019561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37029 (local address 127.12.117.129:57895)
0810 07:19:07.020192 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.020232 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.021504 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.022073 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.022080 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.022164 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:07.022628 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.022640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.023494 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.023497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.024375 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.024384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.024885 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.024890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.025057 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.025640 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:07.025642 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:07.025695 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:07.025775 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:07.026038 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:07.026047 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:07.026053 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.026165 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.026440 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":111,"threads_started":1}
I20260810 07:19:07.027966 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.130:0
--local_ip_for_outbound_sockets=127.12.117.130
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--builtin_ntp_servers=127.12.117.148:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:07.028566 14722 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37029
I20260810 07:19:07.028788 14722 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:07.029297 14722 heartbeater.cc:511] Master 127.12.117.190:37029 requested a full tablet report, sending...
I20260810 07:19:07.030890 14402 ts_manager.cc:194] Registered new tserver with Master: 13096522bcd743d7a24d116271f23928 (127.12.117.129:34307)
I20260810 07:19:07.032428 14402 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.129:57895
W20260810 07:19:07.137838 14726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:07.138072 14726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:07.138108 14726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:07.141963 14726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:07.142007 14726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:07.142057 14726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.130
I20260810 07:19:07.146505 14726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.117.130
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.130
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:07.147420 14726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:07.148676 14726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:07.157577 14737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:07.157609 14732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:07.157589 14734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:07.157838 14726 server_base.cc:1034] running on GCE node
I20260810 07:19:07.158270 14726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:07.158855 14726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:07.160009 14726 hybrid_clock.cc:648] HybridClock initialized: now 1786346347159996 us; error 24 us; skew 500 ppm
I20260810 07:19:07.161808 14726 webserver.cc:460] Webserver started at http://127.12.117.130:44941/ using document root <none> and password file <none>
I20260810 07:19:07.162343 14726 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:07.162405 14726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:07.162597 14726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:07.164527 14726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data/instance:
uuid: "bffdf35e90aa49999b19df2820b822d0"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "c9b97b5913efa188c4121d033d412966"
server_key_iv: "e448e893dc5f12b88acf3a571423bf56"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.165068 14726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance:
uuid: "bffdf35e90aa49999b19df2820b822d0"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "c9b97b5913efa188c4121d033d412966"
server_key_iv: "e448e893dc5f12b88acf3a571423bf56"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.168525 14726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:07.170742 14745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.172017 14726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:07.172142 14726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal
uuid: "bffdf35e90aa49999b19df2820b822d0"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "c9b97b5913efa188c4121d033d412966"
server_key_iv: "e448e893dc5f12b88acf3a571423bf56"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.172248 14726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:07.185343 14726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:07.185954 14726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:07.186120 14726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:07.186661 14726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:07.187587 14726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:07.187637 14726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.187688 14726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:07.187718 14726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.195693 14726 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.130:40159
I20260810 07:19:07.195745 14950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.130:40159 every 8 connection(s)
I20260810 07:19:07.196530 14726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/data/info.pb
I20260810 07:19:07.205173 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14726
I20260810 07:19:07.205307 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal/instance
I20260810 07:19:07.205404 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.198255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.130:52229 (local address 127.12.117.190:37029)
0810 07:19:07.198399 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.198403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.199505 (+ 1102us) server_negotiation.cc:407] Connection header received
0810 07:19:07.200930 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.200934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.200991 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.201066 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.201935 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.202469 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.203611 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.203830 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.203919 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.204876 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.204880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.204883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.204896 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.204914 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.204922 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.204993 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.205174 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.205243 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 07:19:07.205646 12758 external_mini_cluster.cc:1428] Setting key e393517339c58ba2ee383729176b034c
I20260810 07:19:07.206223 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.198661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37029 (local address 127.12.117.130:52229)
0810 07:19:07.199363 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.199403 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.200724 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.201242 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.201249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.201330 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:07.201784 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.201798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.202597 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.202600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.203488 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.203497 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.203964 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.203968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.204131 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.204628 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:07.204630 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:07.204666 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:07.204726 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:07.205015 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:07.205024 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:07.205030 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.205149 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.205419 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
I20260810 07:19:07.207309 14952 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37029
I20260810 07:19:07.207535 14952 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:07.207657 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.117.131:0
--local_ip_for_outbound_sockets=127.12.117.131
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--builtin_ntp_servers=127.12.117.148:42177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 07:19:07.208045 14952 heartbeater.cc:511] Master 127.12.117.190:37029 requested a full tablet report, sending...
I20260810 07:19:07.209028 14400 ts_manager.cc:194] Registered new tserver with Master: bffdf35e90aa49999b19df2820b822d0 (127.12.117.130:40159)
I20260810 07:19:07.209748 14400 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.130:52229
W20260810 07:19:07.317709 14958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:07.317939 14958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:07.317979 14958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:07.321914 14958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:07.321965 14958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:07.322033 14958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.117.131
I20260810 07:19:07.326762 14958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:42177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.117.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.117.131
--webserver_port=0
--tserver_master_addrs=127.12.117.190:37029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.117.131
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:07.327826 14958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:07.329144 14958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:07.338169 14969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:07.338192 14971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:07.338300 14958 server_base.cc:1034] running on GCE node
W20260810 07:19:07.338215 14968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:07.338797 14958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:07.339409 14958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:07.340565 14958 hybrid_clock.cc:648] HybridClock initialized: now 1786346347340546 us; error 30 us; skew 500 ppm
I20260810 07:19:07.345873 14958 webserver.cc:460] Webserver started at http://127.12.117.131:39185/ using document root <none> and password file <none>
I20260810 07:19:07.346398 14958 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:07.346464 14958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:07.346670 14958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:07.348623 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data/instance:
uuid: "96aa64031ae543b8a6da7266f7f8e654"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "5bd79c952eb39f4dacf6e98727da9c5e"
server_key_iv: "2d6fc98263df223a85ccf4ae77519307"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.349161 14958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance:
uuid: "96aa64031ae543b8a6da7266f7f8e654"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "5bd79c952eb39f4dacf6e98727da9c5e"
server_key_iv: "2d6fc98263df223a85ccf4ae77519307"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.352679 14958 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:07.354982 14978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.356109 14958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:07.356210 14958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal
uuid: "96aa64031ae543b8a6da7266f7f8e654"
format_stamp: "Formatted at 2026-08-10 07:19:07 on dist-test-slave-gjrz"
server_key: "5bd79c952eb39f4dacf6e98727da9c5e"
server_key_iv: "2d6fc98263df223a85ccf4ae77519307"
server_key_version: "encryptionkey@0"
I20260810 07:19:07.356295 14958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:07.369117 14958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:07.369679 14958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:07.369825 14958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:07.370323 14958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 07:19:07.371168 14958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 07:19:07.371214 14958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.371281 14958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 07:19:07.371315 14958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:07.379554 14958 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.131:36063
I20260810 07:19:07.379608 15171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.131:36063 every 8 connection(s)
I20260810 07:19:07.380358 14958 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/data/info.pb
I20260810 07:19:07.384963 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 14958
I20260810 07:19:07.385070 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal/instance
I20260810 07:19:07.385361 12758 external_mini_cluster.cc:1428] Setting key 71fdb6bf0499b56786dcc3ad0df0b674
I20260810 07:19:07.388999 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.381983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.131:57065 (local address 127.12.117.190:37029)
0810 07:19:07.382120 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.382124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.383050 (+ 926us) server_negotiation.cc:407] Connection header received
0810 07:19:07.384367 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.384370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.384417 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.384486 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.385526 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.386117 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.387383 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.387649 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.387729 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.388651 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.388653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.388656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.388665 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.388681 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.388687 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.388736 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.388847 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.388893 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 07:19:07.389778 15176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.382335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37029 (local address 127.12.117.131:57065)
0810 07:19:07.382922 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.382958 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.384192 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.384630 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.384636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.384719 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:07.385365 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.385386 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.386249 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.386253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.387233 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.387246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.387765 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.387768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.387912 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.388429 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 07:19:07.388431 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:07.388469 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 07:19:07.388529 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:07.388746 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:07.388750 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:07.388754 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.388838 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.389042 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
I20260810 07:19:07.390753 15173 heartbeater.cc:347] Connected to a master server at 127.12.117.190:37029
I20260810 07:19:07.390944 15173 heartbeater.cc:464] Registering TS with master...
I20260810 07:19:07.391427 15173 heartbeater.cc:511] Master 127.12.117.190:37029 requested a full tablet report, sending...
I20260810 07:19:07.392315 14400 ts_manager.cc:194] Registered new tserver with Master: 96aa64031ae543b8a6da7266f7f8e654 (127.12.117.131:36063)
I20260810 07:19:07.392838 14400 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.117.131:57065
I20260810 07:19:07.398780 12758 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 07:19:07.404202 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.400666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42168 (local address 127.12.117.190:37029)
0810 07:19:07.400799 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.400802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.401046 (+ 244us) server_negotiation.cc:407] Connection header received
0810 07:19:07.401152 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.401155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.401197 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.401262 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:07.401536 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.402040 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.402852 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.403077 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.403162 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.403830 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:07.403834 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:07.403837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:07.403849 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:07.403868 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:07.403875 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:07.403932 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.404029 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.404081 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 07:19:07.406654 14400 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 07:19:07.409611 14400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 07:19:07.423099 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38584 (local address 127.12.117.131:36063)
0810 07:19:07.418023 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.418034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.418058 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:07.418287 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.418293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.418498 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.418643 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:07.419196 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420064 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.421595 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422413 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.422533 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.422765 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.422850 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.422942 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":84,"threads_started":1}
I20260810 07:19:07.423097 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:34307 (local address 127.0.0.1:48472)
0810 07:19:07.418297 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.418310 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.418399 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.418827 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.418833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.418858 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:07.419051 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.419058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420442 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.420445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.421558 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.421567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422558 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.422562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.422672 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.422709 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.422806 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.422948 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":29,"spinlock_wait_cycles":15360,"thread_start_us":81,"threads_started":1}
I20260810 07:19:07.423442 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.12.117.130:40159)
0810 07:19:07.418668 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.418680 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.418995 (+ 315us) server_negotiation.cc:407] Connection header received
0810 07:19:07.419060 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.419064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.419224 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.419388 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:07.419597 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420600 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.421952 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422709 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.422823 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.422984 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.423172 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.423293 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":141,"threads_started":1}
I20260810 07:19:07.423453 15188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:40159 (local address 127.0.0.1:48430)
0810 07:19:07.418805 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.418817 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.418907 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.419367 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.419370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.419389 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:07.419484 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.419490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420700 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.420703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.421797 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.421808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422930 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.422933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.423022 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.423043 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.423107 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.423335 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"spinlock_wait_cycles":768,"thread_start_us":99,"threads_started":1}
I20260810 07:19:07.423684 15187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:36063 (local address 127.0.0.1:38584)
0810 07:19:07.417941 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.417989 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.418115 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.418850 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.418852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.418865 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:07.419051 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.419058 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420165 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.420168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.421451 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.421463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422609 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.422613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.422701 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.422727 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.422823 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.423087 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":179,"spinlock_wait_cycles":15872,"thread_start_us":75,"threads_started":1}
I20260810 07:19:07.424211 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.417848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.12.117.129:34307)
0810 07:19:07.418384 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.418395 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.418419 (+ 24us) server_negotiation.cc:407] Connection header received
0810 07:19:07.418492 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.418497 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.418685 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.418828 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:07.419170 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.420331 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.421672 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.422413 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.423769 (+ 1356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.423934 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.423977 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.424062 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":154,"threads_started":1}
I20260810 07:19:07.426308 15064 tablet_service.cc:1467] Processing CreateTablet for tablet 53650c5fc6744cfea456714c967898f3 (DEFAULT_TABLE table=test-table [id=b297de59cdae4b51ba8b77f20b92f0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:07.426357 14828 tablet_service.cc:1467] Processing CreateTablet for tablet 53650c5fc6744cfea456714c967898f3 (DEFAULT_TABLE table=test-table [id=b297de59cdae4b51ba8b77f20b92f0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:07.426608 14609 tablet_service.cc:1467] Processing CreateTablet for tablet 53650c5fc6744cfea456714c967898f3 (DEFAULT_TABLE table=test-table [id=b297de59cdae4b51ba8b77f20b92f0e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 07:19:07.427256 15064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53650c5fc6744cfea456714c967898f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:07.427441 14828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53650c5fc6744cfea456714c967898f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:07.427441 14609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53650c5fc6744cfea456714c967898f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:07.433138 15202 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Bootstrap starting.
I20260810 07:19:07.434832 15204 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Bootstrap starting.
I20260810 07:19:07.435596 15203 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Bootstrap starting.
I20260810 07:19:07.435788 15202 tablet_bootstrap.cc:654] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:07.436470 15202 log.cc:824] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:07.436846 15204 tablet_bootstrap.cc:654] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:07.437629 15203 tablet_bootstrap.cc:654] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:07.437649 15204 log.cc:824] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:07.438135 15202 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: No bootstrap required, opened a new log
I20260810 07:19:07.438331 15202 ts_tablet_manager.cc:1397] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 07:19:07.438397 15203 log.cc:824] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:07.439611 15204 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: No bootstrap required, opened a new log
I20260810 07:19:07.439899 15204 ts_tablet_manager.cc:1397] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 07:19:07.440524 15203 tablet_bootstrap.cc:492] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: No bootstrap required, opened a new log
I20260810 07:19:07.440704 15203 ts_tablet_manager.cc:1397] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 07:19:07.441805 15202 raft_consensus.cc:348] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.442024 15202 raft_consensus.cc:374] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:07.442118 15202 raft_consensus.cc:729] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96aa64031ae543b8a6da7266f7f8e654, State: Initialized, Role: FOLLOWER
I20260810 07:19:07.442529 15202 consensus_queue.cc:260] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.443099 15173 heartbeater.cc:503] Master 127.12.117.190:37029 was elected leader, sending a full tablet report...
I20260810 07:19:07.443478 15202 ts_tablet_manager.cc:1428] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 07:19:07.443985 15204 raft_consensus.cc:348] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.444242 15204 raft_consensus.cc:374] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:07.444316 15204 raft_consensus.cc:729] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13096522bcd743d7a24d116271f23928, State: Initialized, Role: FOLLOWER
I20260810 07:19:07.444737 15204 consensus_queue.cc:260] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.444708 15203 raft_consensus.cc:348] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.444995 15203 raft_consensus.cc:374] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:07.445104 15203 raft_consensus.cc:729] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bffdf35e90aa49999b19df2820b822d0, State: Initialized, Role: FOLLOWER
I20260810 07:19:07.445343 14722 heartbeater.cc:503] Master 127.12.117.190:37029 was elected leader, sending a full tablet report...
I20260810 07:19:07.445590 15203 consensus_queue.cc:260] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.445696 15204 ts_tablet_manager.cc:1428] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 07:19:07.446250 14952 heartbeater.cc:503] Master 127.12.117.190:37029 was elected leader, sending a full tablet report...
I20260810 07:19:07.446467 15203 ts_tablet_manager.cc:1428] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260810 07:19:07.447386 14953 tablet.cc:2369] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:19:07.518505 14723 tablet.cc:2369] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 07:19:07.631331 15175 tablet.cc:2369] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:07.766567 15211 raft_consensus.cc:482] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 07:19:07.766785 15211 raft_consensus.cc:504] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.767966 15211 leader_election.cc:288] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96aa64031ae543b8a6da7266f7f8e654 (127.12.117.131:36063), bffdf35e90aa49999b19df2820b822d0 (127.12.117.130:40159)
I20260810 07:19:07.771797 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.768289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:42587 (local address 127.12.117.131:36063)
0810 07:19:07.768416 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.768420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.768671 (+ 251us) server_negotiation.cc:407] Connection header received
0810 07:19:07.768807 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.768809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.768852 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.768915 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:07.769209 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.769760 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.770848 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.771247 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.771380 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.771429 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.771619 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.771690 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:07.771806 15199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.768319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.117.129:45471 (local address 127.12.117.130:40159)
0810 07:19:07.768429 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.768433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.768757 (+ 324us) server_negotiation.cc:407] Connection header received
0810 07:19:07.768879 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.768882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.768928 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.768989 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 07:19:07.769294 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.769787 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.770846 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.771237 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.771364 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.771416 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.771628 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.771696 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:07.771817 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.768200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.131:36063 (local address 127.12.117.129:42587)
0810 07:19:07.768563 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.768586 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.768701 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.768931 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.768934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.768951 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:07.769083 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.769090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.769865 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.769868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.770727 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.770736 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.771371 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.771375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.771504 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.771523 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.771588 (+ 65us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.771654 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":34,"thread_start_us":100,"threads_started":1}
I20260810 07:19:07.771823 15213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.768200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.130:40159 (local address 127.12.117.129:45471)
0810 07:19:07.768663 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.768675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:07.768782 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:07.769005 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:07.769008 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:07.769023 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 07:19:07.769173 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.769180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.769895 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.769898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.770732 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:07.770741 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.771359 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.771362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:07.771517 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.771536 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 07:19:07.771600 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 07:19:07.771666 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":232,"thread_start_us":49,"threads_started":1}
I20260810 07:19:07.772478 15097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53650c5fc6744cfea456714c967898f3" candidate_uuid: "13096522bcd743d7a24d116271f23928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96aa64031ae543b8a6da7266f7f8e654" is_pre_election: true
I20260810 07:19:07.772585 14862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53650c5fc6744cfea456714c967898f3" candidate_uuid: "13096522bcd743d7a24d116271f23928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bffdf35e90aa49999b19df2820b822d0" is_pre_election: true
I20260810 07:19:07.772769 15097 raft_consensus.cc:2454] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13096522bcd743d7a24d116271f23928 in term 0.
I20260810 07:19:07.772846 14862 raft_consensus.cc:2454] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13096522bcd743d7a24d116271f23928 in term 0.
I20260810 07:19:07.773262 14529 leader_election.cc:302] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13096522bcd743d7a24d116271f23928, 96aa64031ae543b8a6da7266f7f8e654; no voters:
I20260810 07:19:07.773541 15211 raft_consensus.cc:2781] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 07:19:07.773629 15211 raft_consensus.cc:482] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 07:19:07.773677 15211 raft_consensus.cc:3037] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:07.774717 15211 raft_consensus.cc:504] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.775022 15211 leader_election.cc:288] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [CANDIDATE]: Term 1 election: Requested vote from peers 96aa64031ae543b8a6da7266f7f8e654 (127.12.117.131:36063), bffdf35e90aa49999b19df2820b822d0 (127.12.117.130:40159)
I20260810 07:19:07.775334 14862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53650c5fc6744cfea456714c967898f3" candidate_uuid: "13096522bcd743d7a24d116271f23928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bffdf35e90aa49999b19df2820b822d0"
I20260810 07:19:07.775362 15097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53650c5fc6744cfea456714c967898f3" candidate_uuid: "13096522bcd743d7a24d116271f23928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96aa64031ae543b8a6da7266f7f8e654"
I20260810 07:19:07.775439 14862 raft_consensus.cc:3037] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:07.775502 15097 raft_consensus.cc:3037] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:07.776543 15097 raft_consensus.cc:2454] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13096522bcd743d7a24d116271f23928 in term 1.
I20260810 07:19:07.776623 14862 raft_consensus.cc:2454] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13096522bcd743d7a24d116271f23928 in term 1.
I20260810 07:19:07.776805 14529 leader_election.cc:302] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13096522bcd743d7a24d116271f23928, 96aa64031ae543b8a6da7266f7f8e654; no voters:
I20260810 07:19:07.776960 15211 raft_consensus.cc:2781] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:07.777160 15211 raft_consensus.cc:686] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [term 1 LEADER]: Becoming Leader. State: Replica: 13096522bcd743d7a24d116271f23928, State: Running, Role: LEADER
I20260810 07:19:07.777477 15211 consensus_queue.cc:237] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } }
I20260810 07:19:07.780336 14400 catalog_manager.cc:5495] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13096522bcd743d7a24d116271f23928 (127.12.117.129). New cstate: current_term: 1 leader_uuid: "13096522bcd743d7a24d116271f23928" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13096522bcd743d7a24d116271f23928" member_type: VOTER last_known_addr { host: "127.12.117.129" port: 34307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260810 07:19:07.869252 15197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.866248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48482 (local address 127.12.117.129:34307)
0810 07:19:07.866366 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:07.866370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:07.866401 (+ 31us) server_negotiation.cc:407] Connection header received
0810 07:19:07.866476 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:07.866478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:07.866535 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:07.866613 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:07.866792 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.867344 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.868111 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:07.868312 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:07.868377 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:07.868592 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:07.868978 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:07.869060 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:07.869101 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 07:19:07.869144 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:19:07.874665 15097 raft_consensus.cc:1261] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 1 FOLLOWER]: Refusing update from remote peer 13096522bcd743d7a24d116271f23928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:07.874675 14862 raft_consensus.cc:1261] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 1 FOLLOWER]: Refusing update from remote peer 13096522bcd743d7a24d116271f23928: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:07.875314 15216 consensus_queue.cc:1035] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96aa64031ae543b8a6da7266f7f8e654" member_type: VOTER last_known_addr { host: "127.12.117.131" port: 36063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:07.875495 15211 consensus_queue.cc:1035] T 53650c5fc6744cfea456714c967898f3 P 13096522bcd743d7a24d116271f23928 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bffdf35e90aa49999b19df2820b822d0" member_type: VOTER last_known_addr { host: "127.12.117.130" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:07.881758 15221 mvcc.cc:205] Tried to move back new op lower bound from 7316874640887988224 to 7316874640498761728. Current Snapshot: MvccSnapshot[applied={T|T < 7316874640887988224}]
I20260810 07:19:07.882174 15223 mvcc.cc:205] Tried to move back new op lower bound from 7316874640887988224 to 7316874640498761728. Current Snapshot: MvccSnapshot[applied={T|T < 7316874640887988224}]
I20260810 07:19:07.885003 15224 mvcc.cc:205] Tried to move back new op lower bound from 7316874640887988224 to 7316874640498761728. Current Snapshot: MvccSnapshot[applied={T|T < 7316874640887988224}]
I20260810 07:19:07.891305 14400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:07.891450 14400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 07:19:07.893639 14400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3eed6c58ea4844399ddcaf7230c52c22: Sending DeleteTablet for 3 replicas of tablet 53650c5fc6744cfea456714c967898f3
I20260810 07:19:07.894286 15064 tablet_service.cc:1514] Processing DeleteTablet for tablet 53650c5fc6744cfea456714c967898f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:07 UTC) from {username='slave'} at 127.0.0.1:38584
I20260810 07:19:07.894326 14828 tablet_service.cc:1514] Processing DeleteTablet for tablet 53650c5fc6744cfea456714c967898f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:07 UTC) from {username='slave'} at 127.0.0.1:48430
I20260810 07:19:07.894363 14609 tablet_service.cc:1514] Processing DeleteTablet for tablet 53650c5fc6744cfea456714c967898f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 07:19:07 UTC) from {username='slave'} at 127.0.0.1:48472
I20260810 07:19:07.894877 15242 tablet_replica.cc:330] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: stopping tablet replica
I20260810 07:19:07.894869 15243 tablet_replica.cc:330] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: stopping tablet replica
I20260810 07:19:07.895174 15242 raft_consensus.cc:2229] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:07.895174 15243 raft_consensus.cc:2229] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 07:19:07.895519 15242 raft_consensus.cc:2258] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:07.895520 15243 raft_consensus.cc:2258] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 07:19:07.895682 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14503
I20260810 07:19:07.907797 15243 ts_tablet_manager.cc:1905] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:07.911353 15243 ts_tablet_manager.cc:1918] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:07.911492 15243 log.cc:1196] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-1/wal/wals/53650c5fc6744cfea456714c967898f3
I20260810 07:19:07.911893 15243 ts_tablet_manager.cc:1939] T 53650c5fc6744cfea456714c967898f3 P bffdf35e90aa49999b19df2820b822d0: Deleting consensus metadata
I20260810 07:19:07.913386 14379 catalog_manager.cc:4841] TS bffdf35e90aa49999b19df2820b822d0 (127.12.117.130:40159): tablet 53650c5fc6744cfea456714c967898f3 (table test-table [id=b297de59cdae4b51ba8b77f20b92f0e3]) successfully deleted
I20260810 07:19:07.914196 15242 ts_tablet_manager.cc:1905] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 07:19:07.916313 15242 ts_tablet_manager.cc:1918] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 07:19:07.916405 15242 log.cc:1196] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Deleting WAL directory at /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786346097807571-12758-0/minicluster-data/ts-2/wal/wals/53650c5fc6744cfea456714c967898f3
I20260810 07:19:07.916692 15242 ts_tablet_manager.cc:1939] T 53650c5fc6744cfea456714c967898f3 P 96aa64031ae543b8a6da7266f7f8e654: Deleting consensus metadata
I20260810 07:19:07.917718 14382 catalog_manager.cc:4841] TS 96aa64031ae543b8a6da7266f7f8e654 (127.12.117.131:36063): tablet 53650c5fc6744cfea456714c967898f3 (table test-table [id=b297de59cdae4b51ba8b77f20b92f0e3]) successfully deleted
W20260810 07:19:07.920797 14380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.117.129:34307 (error 108)
I20260810 07:19:07.920908 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14726
I20260810 07:19:07.929318 15195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:07.927410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.129:34307 (local address 127.0.0.1:48496)
0810 07:19:07.929134 (+ 1724us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:07.929221 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.129:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1622}
W20260810 07:19:07.930677 14380 catalog_manager.cc:4584] TS 13096522bcd743d7a24d116271f23928 (127.12.117.129:34307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 53650c5fc6744cfea456714c967898f3: Network error: Client connection negotiation failed: client connection to 127.12.117.129:34307: connect: Connection refused (error 111)
I20260810 07:19:07.937333 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14958
I20260810 07:19:07.948791 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 14354
2026-08-10T07:19:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1360 ms)
[----------] 6 tests from EncryptionPolicyTest (10780 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:08 dist-test-slave-gjrz krb5kdc[15251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:08 dist-test-slave-gjrz krb5kdc[15251](info): set up 2 sockets
Aug 10 07:19:08 dist-test-slave-gjrz krb5kdc[15251](info): commencing operation
krb5kdc: starting...
W20260810 07:19:09.109339 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:09.219926 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:09.325446 15269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:09.325721 15269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:09.325760 15269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:09.329636 15269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:09.329692 15269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:09.329710 15269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:09.329725 15269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:09.329743 15269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:09.334698 15269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:09.335654 15269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:09.336591 15269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:09.344553 15269 server_base.cc:1034] running on GCE node
W20260810 07:19:09.344498 15280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:09.344493 15279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:09.344519 15284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:09.345252 15269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:09.346211 15269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:09.347314 15269 hybrid_clock.cc:648] HybridClock initialized: now 1786346349347295 us; error 32 us; skew 500 ppm
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346349, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:09.349788 15269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:09.350584 15269 webserver.cc:460] Webserver started at http://127.0.0.1:39371/ using document root <none> and password file <none>
I20260810 07:19:09.351064 15269 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:09.351154 15269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:09.351382 15269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:09.353261 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "286991a204ab4b12863ade500f1d41bb"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-gjrz"
server_key: "286b66dc8f98fe6565cb7d13f1cae571"
server_key_iv: "926524252dc4e4f048a0d725a1edacac"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.353768 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "286991a204ab4b12863ade500f1d41bb"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-gjrz"
server_key: "286b66dc8f98fe6565cb7d13f1cae571"
server_key_iv: "926524252dc4e4f048a0d725a1edacac"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.357221 15269 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 07:19:09.359434 15295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:09.360575 15269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:09.360673 15269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "286991a204ab4b12863ade500f1d41bb"
format_stamp: "Formatted at 2026-08-10 07:19:09 on dist-test-slave-gjrz"
server_key: "286b66dc8f98fe6565cb7d13f1cae571"
server_key_iv: "926524252dc4e4f048a0d725a1edacac"
server_key_version: "encryptionkey@0"
I20260810 07:19:09.360767 15269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:09.378101 15269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:09.380918 15269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:09.381088 15269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:09.387521 15269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39955
I20260810 07:19:09.387555 15382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39955 every 8 connection(s)
I20260810 07:19:09.388360 15269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:09.390895 15384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:09.394958 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15269
I20260810 07:19:09.395080 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:09.395366 12758 external_mini_cluster.cc:1428] Setting key 02414cf6a5b2d44f4fe15739dbe0cf5b
I20260810 07:19:09.396358 15384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb: Bootstrap starting.
I20260810 07:19:09.398448 15384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:09.399343 15384 log.cc:824] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:09.401268 15384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb: No bootstrap required, opened a new log
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:19:09.405129 15384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260810 07:19:09.405333 15384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:09.405403 15384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286991a204ab4b12863ade500f1d41bb, State: Initialized, Role: FOLLOWER
I20260810 07:19:09.405875 15384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260810 07:19:09.406030 15384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:09.406096 15384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:09.406193 15384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:09.407127 15384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260810 07:19:09.407472 15384 leader_election.cc:302] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 286991a204ab4b12863ade500f1d41bb; no voters:
I20260810 07:19:09.407753 15384 leader_election.cc:288] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:09.407830 15394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:09.408001 15394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [term 1 LEADER]: Becoming Leader. State: Replica: 286991a204ab4b12863ade500f1d41bb, State: Running, Role: LEADER
I20260810 07:19:09.408543 15394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } }
I20260810 07:19:09.408991 15384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:09.409607 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:09.396702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48461 (local address 127.0.0.1:39955)
0810 07:19:09.397059 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:09.397081 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:09.397113 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:09.397713 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:09.397728 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:09.398038 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:09.398304 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:09.399192 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:09.400221 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:09.401352 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:09.401845 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:09.401971 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:09.404516 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:09.404522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:09.404534 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:09.404560 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:09.406361 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:09.406840 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:09.406847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:09.406853 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:09.407008 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:09.407340 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:09.407343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:09.407345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:09.407755 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:09.408518 (+ 763us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:09.408655 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:19:09.408858 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
I20260810 07:19:09.409739 15394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286991a204ab4b12863ade500f1d41bb. Latest consensus state: current_term: 1 leader_uuid: "286991a204ab4b12863ade500f1d41bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } } }
I20260810 07:19:09.409930 15394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:09.410398 15405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:09.411072 15395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "286991a204ab4b12863ade500f1d41bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286991a204ab4b12863ade500f1d41bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39955 } } }
I20260810 07:19:09.411166 15395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:09.412858 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:09.415055 15405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:09.420614 15405 catalog_manager.cc:1347] Generated new cluster ID: 3107a3e83c81429e93714d4bb07c02ea
I20260810 07:19:09.420671 15405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:09.424269 15389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:09.414862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46803 (local address 127.0.0.1:39955)
0810 07:19:09.414984 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:09.414988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:09.416969 (+ 1981us) server_negotiation.cc:407] Connection header received
0810 07:19:09.417071 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:09.417074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:09.417112 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:09.417173 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:09.417774 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:09.418601 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:09.419594 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:09.419951 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:09.420060 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:09.421377 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:09.421381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:09.421384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:09.421411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:09.422639 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:09.423091 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:09.423095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:09.423096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:09.423164 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:09.423483 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:09.423486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:09.423487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:09.423641 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:09.423763 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:09.424048 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 07:19:09.424133 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:09.435329 15405 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:09.436252 15405 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:09.441710 15405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 286991a204ab4b12863ade500f1d41bb: Generated new TSK 0
I20260810 07:19:09.442279 15405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:09.447985 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15269
W20260810 07:19:09.467425 15247 connection.cc:535] client connection to 127.0.0.1:39955 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39955): Connection reset by peer (error 104)
2026-08-10T07:19:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15430](info): set up 2 sockets
Aug 10 07:19:09 dist-test-slave-gjrz krb5kdc[15430](info): commencing operation
krb5kdc: starting...
W20260810 07:19:10.643360 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:10 dist-test-slave-gjrz krb5kdc[15430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:10.760731 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:10.867933 15455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:10.868181 15455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:10.868227 15455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:10.872190 15455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:10.872242 15455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:10.872260 15455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:10.872289 15455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:10.872315 15455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:10.877404 15455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:10.878322 15455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:10.879293 15455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:10.887243 15465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:10.887326 15469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:10.887427 15463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:10.887450 15455 server_base.cc:1034] running on GCE node
I20260810 07:19:10.888219 15455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:10.889238 15455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:10.890412 15455 hybrid_clock.cc:648] HybridClock initialized: now 1786346350890386 us; error 39 us; skew 500 ppm
Aug 10 07:19:10 dist-test-slave-gjrz krb5kdc[15430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:10.893029 15455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:10.893821 15455 webserver.cc:460] Webserver started at http://127.0.0.1:43753/ using document root <none> and password file <none>
I20260810 07:19:10.894305 15455 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:10.894376 15455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:10.894556 15455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:10.896579 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "30845d48055b475fa808f673d5bbae5a"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-gjrz"
server_key: "239641d1de10b3c1ff3aed074eaed75b"
server_key_iv: "eb1192e6acf754a8cc4eda8d6d0f91ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.897094 15455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "30845d48055b475fa808f673d5bbae5a"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-gjrz"
server_key: "239641d1de10b3c1ff3aed074eaed75b"
server_key_iv: "eb1192e6acf754a8cc4eda8d6d0f91ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.900599 15455 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:10.902858 15479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:10.904038 15455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:10.904152 15455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "30845d48055b475fa808f673d5bbae5a"
format_stamp: "Formatted at 2026-08-10 07:19:10 on dist-test-slave-gjrz"
server_key: "239641d1de10b3c1ff3aed074eaed75b"
server_key_iv: "eb1192e6acf754a8cc4eda8d6d0f91ff"
server_key_version: "encryptionkey@0"
I20260810 07:19:10.904233 15455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:10.915252 15455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:10.917655 15455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:10.917788 15455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:10.923921 15455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46367
I20260810 07:19:10.923938 15564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46367 every 8 connection(s)
I20260810 07:19:10.924767 15455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:10.925776 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15455
I20260810 07:19:10.925913 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:10.926226 12758 external_mini_cluster.cc:1428] Setting key 09bc6bfbf43a99ebd510c72d6484fd71
I20260810 07:19:10.927814 15565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:10.933521 15565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a: Bootstrap starting.
I20260810 07:19:10.934517 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:10.927481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46631 (local address 127.0.0.1:46367)
0810 07:19:10.927844 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:10.927862 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:10.927912 (+ 50us) server_negotiation.cc:407] Connection header received
0810 07:19:10.928515 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:10.928530 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:10.928850 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:10.929114 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:10.929461 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.930404 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.931569 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.932013 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.932128 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:10.932950 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:10.932954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:10.932966 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:10.933305 (+ 339us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:10.933355 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:10.933376 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:10.933469 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:10.933687 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:19:10.933821 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260810 07:19:10.935806 15565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:10.936617 15565 log.cc:824] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:10.937412 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:10.938741 15565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a: No bootstrap required, opened a new log
I20260810 07:19:10.943076 15565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260810 07:19:10.943317 15575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:10.939609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55079 (local address 127.0.0.1:46367)
0810 07:19:10.939711 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:10.939715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:10.939861 (+ 146us) server_negotiation.cc:407] Connection header received
0810 07:19:10.939969 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:10.939971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:10.940006 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:10.940066 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:10.940321 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.940800 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.941885 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:10.942095 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:10.942160 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:10.942901 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:10.942903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:10.942905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:10.942916 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:10.942938 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:10.942947 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:10.943010 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:10.943148 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:19:10.943173 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:10.943372 15565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:10.943439 15565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30845d48055b475fa808f673d5bbae5a, State: Initialized, Role: FOLLOWER
I20260810 07:19:10.943895 15565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260810 07:19:10.944056 15565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:10.944125 15565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:10.944223 15565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:10.945317 15565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260810 07:19:10.945695 15565 leader_election.cc:302] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30845d48055b475fa808f673d5bbae5a; no voters:
I20260810 07:19:10.946025 15565 leader_election.cc:288] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:10.946127 15585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:10.946350 15585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [term 1 LEADER]: Becoming Leader. State: Replica: 30845d48055b475fa808f673d5bbae5a, State: Running, Role: LEADER
I20260810 07:19:10.946815 15585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260810 07:19:10.947207 15565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:10.948334 15591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30845d48055b475fa808f673d5bbae5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } } }
I20260810 07:19:10.948460 15591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:10.948838 15604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:10.949345 15590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30845d48055b475fa808f673d5bbae5a. Latest consensus state: current_term: 1 leader_uuid: "30845d48055b475fa808f673d5bbae5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30845d48055b475fa808f673d5bbae5a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } } }
I20260810 07:19:10.949514 15590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:10.951978 15604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:10.957415 15604 catalog_manager.cc:1347] Generated new cluster ID: 3300c9cde64d44f1a7cbe27fbb95c059
I20260810 07:19:10.957468 15604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:10.965193 15604 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:10.965996 15604 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:10.971558 15604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30845d48055b475fa808f673d5bbae5a: Generated new TSK 0
I20260810 07:19:10.972112 15604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:10.980371 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15455
W20260810 07:19:10.991743 15426 connection.cc:535] client connection to 127.0.0.1:46367 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46367): Connection reset by peer (error 104)
2026-08-10T07:19:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:11 dist-test-slave-gjrz krb5kdc[15619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:11 dist-test-slave-gjrz krb5kdc[15619](info): set up 2 sockets
Aug 10 07:19:11 dist-test-slave-gjrz krb5kdc[15619](info): commencing operation
krb5kdc: starting...
W20260810 07:19:12.143939 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:12 dist-test-slave-gjrz krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:12.257648 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:12.364884 15640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:12.365119 15640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:12.365160 15640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:12.369057 15640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:12.369109 15640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:12.369127 15640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:12.369143 15640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:12.369158 15640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:12.374221 15640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:12.375105 15640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:12.376071 15640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:12.383893 15649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:12.383903 15646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:12.383971 15645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:12.384157 15640 server_base.cc:1034] running on GCE node
I20260810 07:19:12.384653 15640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:12.385612 15640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:12.386770 15640 hybrid_clock.cc:648] HybridClock initialized: now 1786346352386751 us; error 30 us; skew 500 ppm
Aug 10 07:19:12 dist-test-slave-gjrz krb5kdc[15619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:12.392259 15640 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:12.393062 15640 webserver.cc:460] Webserver started at http://127.0.0.1:35315/ using document root <none> and password file <none>
I20260810 07:19:12.393539 15640 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:12.393594 15640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:12.393754 15640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:12.395630 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "f75c938fc8104f7191f0adab0494fae2"
format_stamp: "Formatted at 2026-08-10 07:19:12 on dist-test-slave-gjrz"
server_key: "5d77a1cd974b98a731d0a9f69ed0a057"
server_key_iv: "e1524ffb1973b762adb0f556e79fcb79"
server_key_version: "encryptionkey@0"
I20260810 07:19:12.396126 15640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "f75c938fc8104f7191f0adab0494fae2"
format_stamp: "Formatted at 2026-08-10 07:19:12 on dist-test-slave-gjrz"
server_key: "5d77a1cd974b98a731d0a9f69ed0a057"
server_key_iv: "e1524ffb1973b762adb0f556e79fcb79"
server_key_version: "encryptionkey@0"
I20260810 07:19:12.399494 15640 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:19:12.401647 15657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:12.402772 15640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:12.402877 15640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "f75c938fc8104f7191f0adab0494fae2"
format_stamp: "Formatted at 2026-08-10 07:19:12 on dist-test-slave-gjrz"
server_key: "5d77a1cd974b98a731d0a9f69ed0a057"
server_key_iv: "e1524ffb1973b762adb0f556e79fcb79"
server_key_version: "encryptionkey@0"
I20260810 07:19:12.402974 15640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:12.407192 15640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:12.409637 15640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:12.409785 15640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:12.416067 15640 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44187
I20260810 07:19:12.416076 15749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44187 every 8 connection(s)
I20260810 07:19:12.416880 15640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:12.419442 15750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:12.422446 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15640
I20260810 07:19:12.422559 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:12.422808 12758 external_mini_cluster.cc:1428] Setting key 775d8be7bd61b28d1bfa83dcb4fa8a7d
I20260810 07:19:12.424775 15750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2: Bootstrap starting.
I20260810 07:19:12.426918 15750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:12.427672 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:12.424071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49479 (local address 127.0.0.1:44187)
0810 07:19:12.424483 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:12.424496 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:12.424537 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:12.425162 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:12.425180 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:12.425454 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:12.425750 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:12.426478 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:12.426482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:12.426493 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:12.426504 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:12.426566 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:12.426585 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:12.426676 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:12.426805 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 07:19:12.426951 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260810 07:19:12.427675 15750 log.cc:824] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:12.429414 15750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2: No bootstrap required, opened a new log
I20260810 07:19:12.430495 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:12.433032 15750 raft_consensus.cc:348] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260810 07:19:12.433285 15750 raft_consensus.cc:374] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:12.433344 15750 raft_consensus.cc:729] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75c938fc8104f7191f0adab0494fae2, State: Initialized, Role: FOLLOWER
I20260810 07:19:12.433727 15750 consensus_queue.cc:260] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260810 07:19:12.433883 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:12.432397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43659 (local address 127.0.0.1:44187)
0810 07:19:12.432490 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:12.432494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:12.432673 (+ 179us) server_negotiation.cc:407] Connection header received
0810 07:19:12.432771 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:12.432774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:12.432802 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:12.432864 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:12.433537 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:12.433540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:12.433542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:12.433555 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:12.433580 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:12.433591 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:12.433663 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:12.433750 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:19:12.433783 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:19:12.433920 15750 raft_consensus.cc:388] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:12.433991 15750 raft_consensus.cc:482] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:12.434090 15750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:12.435031 15750 raft_consensus.cc:504] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260810 07:19:12.435355 15750 leader_election.cc:302] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f75c938fc8104f7191f0adab0494fae2; no voters:
I20260810 07:19:12.435622 15750 leader_election.cc:288] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:12.435691 15764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:12.435945 15764 raft_consensus.cc:686] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [term 1 LEADER]: Becoming Leader. State: Replica: f75c938fc8104f7191f0adab0494fae2, State: Running, Role: LEADER
I20260810 07:19:12.436250 15764 consensus_queue.cc:237] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } }
I20260810 07:19:12.436942 15750 sys_catalog.cc:564] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:12.437444 15765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f75c938fc8104f7191f0adab0494fae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } } }
I20260810 07:19:12.437482 15766 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f75c938fc8104f7191f0adab0494fae2. Latest consensus state: current_term: 1 leader_uuid: "f75c938fc8104f7191f0adab0494fae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f75c938fc8104f7191f0adab0494fae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44187 } } }
I20260810 07:19:12.437634 15765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:12.437641 15766 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:12.438151 15774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:12.441080 15774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:12.446319 15774 catalog_manager.cc:1347] Generated new cluster ID: f194b4ff966c48e0a68d6ce0e24923d9
I20260810 07:19:12.446368 15774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:12.481343 15774 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:12.481662 15774 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:12.493731 15774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f75c938fc8104f7191f0adab0494fae2: Generated new TSK 0
I20260810 07:19:12.494428 15774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:12.499198 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15640
2026-08-10T07:19:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:12 dist-test-slave-gjrz krb5kdc[15795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:12 dist-test-slave-gjrz krb5kdc[15795](info): set up 2 sockets
Aug 10 07:19:12 dist-test-slave-gjrz krb5kdc[15795](info): commencing operation
krb5kdc: starting...
W20260810 07:19:13.652482 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:13 dist-test-slave-gjrz krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:13.766032 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:13.872293 15811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:13.872525 15811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:13.872588 15811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:13.876467 15811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:13.876520 15811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:13.876546 15811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:13.876569 15811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:13.876605 15811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:13.881649 15811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:13.882630 15811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:13.883556 15811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:13.891732 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:13.891794 15817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:13.891782 15811 server_base.cc:1034] running on GCE node
W20260810 07:19:13.891741 15821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:13.892458 15811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:13.893411 15811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:13.894557 15811 hybrid_clock.cc:648] HybridClock initialized: now 1786346353894538 us; error 39 us; skew 500 ppm
Aug 10 07:19:13 dist-test-slave-gjrz krb5kdc[15795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:13.897028 15811 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:13.897958 15811 webserver.cc:460] Webserver started at http://127.0.0.1:36859/ using document root <none> and password file <none>
I20260810 07:19:13.898587 15811 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:13.898669 15811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:13.898890 15811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:13.901113 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "3b2d621adeb54957a59c67c3cf3360f9"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-gjrz"
server_key: "3c035010ec7c53df7ed38e745ae4a66f"
server_key_iv: "511486eaea3161c3a34ea8e2ee52c5a0"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.901652 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "3b2d621adeb54957a59c67c3cf3360f9"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-gjrz"
server_key: "3c035010ec7c53df7ed38e745ae4a66f"
server_key_iv: "511486eaea3161c3a34ea8e2ee52c5a0"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.905108 15811 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:13.907321 15830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:13.908403 15811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:13.908514 15811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "3b2d621adeb54957a59c67c3cf3360f9"
format_stamp: "Formatted at 2026-08-10 07:19:13 on dist-test-slave-gjrz"
server_key: "3c035010ec7c53df7ed38e745ae4a66f"
server_key_iv: "511486eaea3161c3a34ea8e2ee52c5a0"
server_key_version: "encryptionkey@0"
I20260810 07:19:13.908612 15811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:13.932832 15811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:13.935310 15811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:13.935478 15811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:13.941783 15811 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34197
I20260810 07:19:13.941798 15914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34197 every 8 connection(s)
I20260810 07:19:13.942654 15811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:13.945297 15915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:13.950385 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: Bootstrap starting.
I20260810 07:19:13.951159 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15811
I20260810 07:19:13.951310 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:13.951603 12758 external_mini_cluster.cc:1428] Setting key 16297a3ac65679f554f9a45e70ce8c45
I20260810 07:19:13.953083 15915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:13.953917 15915 log.cc:824] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:13.956158 15915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: No bootstrap required, opened a new log
I20260810 07:19:13.960251 15915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260810 07:19:13.960523 15915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:13.960584 15915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b2d621adeb54957a59c67c3cf3360f9, State: Initialized, Role: FOLLOWER
Aug 10 07:19:13 dist-test-slave-gjrz krb5kdc[15795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:19:13.961035 15915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260810 07:19:13.961311 15915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:13.961437 15915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:13.961565 15915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:13.962577 15915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260810 07:19:13.962894 15915 leader_election.cc:302] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b2d621adeb54957a59c67c3cf3360f9; no voters:
I20260810 07:19:13.963224 15915 leader_election.cc:288] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:13.963325 15923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:13.963531 15923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [term 1 LEADER]: Becoming Leader. State: Replica: 3b2d621adeb54957a59c67c3cf3360f9, State: Running, Role: LEADER
I20260810 07:19:13.964114 15923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260810 07:19:13.964448 15915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:13.965713 15924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b2d621adeb54957a59c67c3cf3360f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } } }
I20260810 07:19:13.965834 15924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:13.966218 15928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b2d621adeb54957a59c67c3cf3360f9. Latest consensus state: current_term: 1 leader_uuid: "3b2d621adeb54957a59c67c3cf3360f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b2d621adeb54957a59c67c3cf3360f9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } } }
I20260810 07:19:13.966409 15928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:13.966796 15921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:13.952633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34659 (local address 127.0.0.1:34197)
0810 07:19:13.953027 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:13.953041 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:13.953086 (+ 45us) server_negotiation.cc:407] Connection header received
0810 07:19:13.953814 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:13.953840 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:13.954273 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:13.954607 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:13.955519 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:13.956957 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:13.957812 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:13.958525 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:13.958702 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:13.961271 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:13.961278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:13.961293 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:13.961322 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:13.962946 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:13.963503 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:13.963509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:13.963516 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:13.963677 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:13.963972 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:13.963975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:13.963977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:13.964473 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:13.964664 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:13.965501 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 07:19:13.965828 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
W20260810 07:19:13.969167 15948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:13.969239 15948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:13.969295 15944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:13.969959 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:13.970161 15944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:13.975661 15944 catalog_manager.cc:1347] Generated new cluster ID: 3844a7b4d6944da7811af1fddb1e9c5c
I20260810 07:19:13.975723 15944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:13.979930 15921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:13.972182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48639 (local address 127.0.0.1:34197)
0810 07:19:13.972281 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:13.972285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:13.972525 (+ 240us) server_negotiation.cc:407] Connection header received
0810 07:19:13.972640 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:13.972643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:13.972688 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:13.972770 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:13.973516 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:13.974303 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:13.975462 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:13.975793 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:13.975895 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:13.977134 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:13.977138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:13.977141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:13.977166 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:13.978410 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:13.978788 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:13.978791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:13.978793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:13.978867 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:13.979119 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:13.979122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:13.979124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:13.979344 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:13.979464 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:13.979682 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 07:19:13.979785 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:13.986683 15944 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:13.987505 15944 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:14.003383 15944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b2d621adeb54957a59c67c3cf3360f9: Generated new TSK 0
I20260810 07:19:14.003999 15944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:14.012405 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15811
2026-08-10T07:19:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1521 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:14 dist-test-slave-gjrz krb5kdc[15966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:14 dist-test-slave-gjrz krb5kdc[15966](info): set up 2 sockets
Aug 10 07:19:14 dist-test-slave-gjrz krb5kdc[15966](info): commencing operation
krb5kdc: starting...
W20260810 07:19:15.175243 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:15 dist-test-slave-gjrz krb5kdc[15966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:15.287858 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:15.394243 15984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:15.394474 15984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:15.394529 15984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:15.398393 15984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:15.398447 15984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:15.398473 15984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:15.398494 15984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:15.398523 15984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:15.403498 15984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:15.404390 15984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:15.405298 15984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:15.413086 15998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:15.413141 15990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:15.413223 15991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:15.413251 15984 server_base.cc:1034] running on GCE node
I20260810 07:19:15.413870 15984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:15.414839 15984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:15.415989 15984 hybrid_clock.cc:648] HybridClock initialized: now 1786346355415976 us; error 30 us; skew 500 ppm
Aug 10 07:19:15 dist-test-slave-gjrz krb5kdc[15966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:15.418423 15984 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:15.419221 15984 webserver.cc:460] Webserver started at http://127.0.0.1:38861/ using document root <none> and password file <none>
I20260810 07:19:15.419732 15984 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:15.419797 15984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:15.419987 15984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:15.421828 15984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "ae7ae0c0c04742a78aeef8e779a6ca76"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-gjrz"
server_key: "1f02247a5b4b0cf5c52704f99399772a"
server_key_iv: "10f7a965c13c824f50927cc5c075f960"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.422359 15984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "ae7ae0c0c04742a78aeef8e779a6ca76"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-gjrz"
server_key: "1f02247a5b4b0cf5c52704f99399772a"
server_key_iv: "10f7a965c13c824f50927cc5c075f960"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.425967 15984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:19:15.428278 16009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:15.429391 15984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:15.429504 15984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "ae7ae0c0c04742a78aeef8e779a6ca76"
format_stamp: "Formatted at 2026-08-10 07:19:15 on dist-test-slave-gjrz"
server_key: "1f02247a5b4b0cf5c52704f99399772a"
server_key_iv: "10f7a965c13c824f50927cc5c075f960"
server_key_version: "encryptionkey@0"
I20260810 07:19:15.429593 15984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:15.442440 15984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:15.444923 15984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:15.445081 15984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:15.451418 15984 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40009
I20260810 07:19:15.451426 16086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40009 every 8 connection(s)
I20260810 07:19:15.452234 15984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:15.452811 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 15984
I20260810 07:19:15.452958 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:15.453239 12758 external_mini_cluster.cc:1428] Setting key 35280e50716126dfef0d2ed3b9b35d00
I20260810 07:19:15.455204 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:15.460990 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76: Bootstrap starting.
I20260810 07:19:15.461571 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.454478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34135 (local address 127.0.0.1:40009)
0810 07:19:15.454860 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.454874 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.454915 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:15.455559 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.455577 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.455886 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.456152 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.456521 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.457495 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.458674 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.459133 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.459240 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.460037 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.460041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.460054 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:15.460395 (+ 341us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.460445 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:15.460466 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.460538 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.460697 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.460879 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":116,"threads_started":1}
I20260810 07:19:15.463192 16088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:15.463949 16088 log.cc:824] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:15.464223 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:15.465783 16088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76: No bootstrap required, opened a new log
I20260810 07:19:15.469091 16088 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260810 07:19:15.469295 16088 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:15.469399 16088 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7ae0c0c04742a78aeef8e779a6ca76, State: Initialized, Role: FOLLOWER
I20260810 07:19:15.469430 16092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:15.466051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56361 (local address 127.0.0.1:40009)
0810 07:19:15.466146 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:15.466150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:15.466331 (+ 181us) server_negotiation.cc:407] Connection header received
0810 07:19:15.466430 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:15.466433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:15.466471 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:15.466533 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:15.466768 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.467241 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.468034 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:15.468270 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:15.468353 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:15.469014 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:15.469018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:15.469021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:15.469033 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:15.469058 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:15.469069 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:15.469125 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:15.469224 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 07:19:15.469270 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 07:19:15.469785 16088 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260810 07:19:15.469911 16088 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:15.469970 16088 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:15.470062 16088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:15.470896 16088 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260810 07:19:15.471199 16088 leader_election.cc:302] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae7ae0c0c04742a78aeef8e779a6ca76; no voters:
I20260810 07:19:15.471469 16088 leader_election.cc:288] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:15.471560 16103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:15.471777 16103 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [term 1 LEADER]: Becoming Leader. State: Replica: ae7ae0c0c04742a78aeef8e779a6ca76, State: Running, Role: LEADER
I20260810 07:19:15.472082 16103 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } }
I20260810 07:19:15.472517 16088 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:15.473554 16108 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } } }
I20260810 07:19:15.473656 16108 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:15.473935 16110 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae7ae0c0c04742a78aeef8e779a6ca76. Latest consensus state: current_term: 1 leader_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7ae0c0c04742a78aeef8e779a6ca76" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40009 } } }
I20260810 07:19:15.474031 16110 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:15.474045 16123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:15.476596 16123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:15.481633 16123 catalog_manager.cc:1347] Generated new cluster ID: 310d99f889c142ca8c2bc161cd323040
I20260810 07:19:15.481684 16123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:15.524881 16123 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:15.525740 16123 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:15.532506 16123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae7ae0c0c04742a78aeef8e779a6ca76: Generated new TSK 0
I20260810 07:19:15.533180 16123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:15.546830 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 15984
2026-08-10T07:19:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1530 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:15 dist-test-slave-gjrz krb5kdc[16142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:15 dist-test-slave-gjrz krb5kdc[16142](info): set up 2 sockets
Aug 10 07:19:15 dist-test-slave-gjrz krb5kdc[16142](info): commencing operation
krb5kdc: starting...
W20260810 07:19:16.705271 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:16 dist-test-slave-gjrz krb5kdc[16142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:16.818113 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46735
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:16.922878 16158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:16.923107 16158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:16.923164 16158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:16.926992 16158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:16.927045 16158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:16.927071 16158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:16.927093 16158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:16.927112 16158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:16.932116 16158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:16.933300 16158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:16.934258 16158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:16.942140 16170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:16.942152 16164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:16.942332 16168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:16.942503 16158 server_base.cc:1034] running on GCE node
I20260810 07:19:16.943012 16158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:16.944027 16158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:16.945184 16158 hybrid_clock.cc:648] HybridClock initialized: now 1786346356945173 us; error 30 us; skew 500 ppm
Aug 10 07:19:16 dist-test-slave-gjrz krb5kdc[16142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346356, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:16.947666 16158 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:16.948465 16158 webserver.cc:460] Webserver started at http://127.0.0.1:44289/ using document root <none> and password file <none>
I20260810 07:19:16.948961 16158 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:16.949024 16158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:16.949210 16158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:16.951076 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "31e6bfe3366e4b4faca73dec80768a06"
format_stamp: "Formatted at 2026-08-10 07:19:16 on dist-test-slave-gjrz"
server_key: "f09f4b19cfadb9187b5f6a76c445d2c8"
server_key_iv: "9efc79f504043feed8a42d0d76dac51c"
server_key_version: "encryptionkey@0"
I20260810 07:19:16.951673 16158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "31e6bfe3366e4b4faca73dec80768a06"
format_stamp: "Formatted at 2026-08-10 07:19:16 on dist-test-slave-gjrz"
server_key: "f09f4b19cfadb9187b5f6a76c445d2c8"
server_key_iv: "9efc79f504043feed8a42d0d76dac51c"
server_key_version: "encryptionkey@0"
I20260810 07:19:16.955016 16158 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:16.957226 16186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:16.958318 16158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:16.958420 16158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "31e6bfe3366e4b4faca73dec80768a06"
format_stamp: "Formatted at 2026-08-10 07:19:16 on dist-test-slave-gjrz"
server_key: "f09f4b19cfadb9187b5f6a76c445d2c8"
server_key_iv: "9efc79f504043feed8a42d0d76dac51c"
server_key_version: "encryptionkey@0"
I20260810 07:19:16.958508 16158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:16.962757 16158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:16.966066 16158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:16.966205 16158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:16.972512 16158 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46735
I20260810 07:19:16.972545 16270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46735 every 8 connection(s)
I20260810 07:19:16.973346 16158 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:16.975965 16272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:16.981060 16272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06: Bootstrap starting.
I20260810 07:19:16.982769 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16158
I20260810 07:19:16.982910 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:16.983217 12758 external_mini_cluster.cc:1428] Setting key dab56133e58793325175405cee6ff8e2
I20260810 07:19:16.983624 16272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:16.984392 16272 log.cc:824] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:16.986163 16272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06: No bootstrap required, opened a new log
I20260810 07:19:16.987936 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.984266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58871 (local address 127.0.0.1:46735)
0810 07:19:16.984694 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.984709 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.984753 (+ 44us) server_negotiation.cc:407] Connection header received
0810 07:19:16.985454 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.985474 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.985742 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.986035 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:16.986823 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:16.986829 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:16.986842 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:16.986853 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:16.986905 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:16.986923 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:16.987014 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.987136 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.987302 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":125,"threads_started":1}
I20260810 07:19:16.989883 16272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } }
I20260810 07:19:16.990127 16272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:16.990187 16272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e6bfe3366e4b4faca73dec80768a06, State: Initialized, Role: FOLLOWER
I20260810 07:19:16.990523 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:16.990587 16272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } }
I20260810 07:19:16.990736 16272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:16.990797 16272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:16.990888 16272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:16.991719 16272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } }
I20260810 07:19:16.992031 16272 leader_election.cc:302] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31e6bfe3366e4b4faca73dec80768a06; no voters:
I20260810 07:19:16.992316 16272 leader_election.cc:288] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:16.992441 16284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:16.992643 16284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [term 1 LEADER]: Becoming Leader. State: Replica: 31e6bfe3366e4b4faca73dec80768a06, State: Running, Role: LEADER
I20260810 07:19:16.992997 16284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } }
I20260810 07:19:16.993503 16272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:16.994319 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:16.992406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59125 (local address 127.0.0.1:46735)
0810 07:19:16.992501 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:16.992505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:16.992879 (+ 374us) server_negotiation.cc:407] Connection header received
0810 07:19:16.993194 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:16.993197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:16.993229 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:16.993305 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:16.993948 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:16.993951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:16.993953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:16.993964 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:16.993989 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:16.994001 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:16.994081 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:16.994174 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 07:19:16.994209 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 07:19:16.994817 16287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31e6bfe3366e4b4faca73dec80768a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } } }
I20260810 07:19:16.994938 16287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:16.995290 16288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31e6bfe3366e4b4faca73dec80768a06. Latest consensus state: current_term: 1 leader_uuid: "31e6bfe3366e4b4faca73dec80768a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31e6bfe3366e4b4faca73dec80768a06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46735 } } }
I20260810 07:19:16.995436 16288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:16.995499 16302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:16.998788 16302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:17.003902 16302 catalog_manager.cc:1347] Generated new cluster ID: c45ce382a8f4411b89409b76e0e3fe45
I20260810 07:19:17.003965 16302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:17.015565 16302 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:17.015843 16302 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:17.020004 16302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31e6bfe3366e4b4faca73dec80768a06: Generated new TSK 0
I20260810 07:19:17.020678 16302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:17.029843 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16158
2026-08-10T07:19:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1487 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:17 dist-test-slave-gjrz krb5kdc[16312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:17 dist-test-slave-gjrz krb5kdc[16312](info): set up 2 sockets
Aug 10 07:19:17 dist-test-slave-gjrz krb5kdc[16312](info): commencing operation
krb5kdc: starting...
W20260810 07:19:18.202313 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:18.314415 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:18.419659 16334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:18.419873 16334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:18.419910 16334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:18.423734 16334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:18.423780 16334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:18.423799 16334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:18.423817 16334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:18.423831 16334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:18.428764 16334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:18.429606 16334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:18.430465 16334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:18.438184 16348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:18.438256 16334 server_base.cc:1034] running on GCE node
W20260810 07:19:18.438174 16347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:18.438156 16350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:18.438859 16334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:18.439844 16334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:18.441015 16334 hybrid_clock.cc:648] HybridClock initialized: now 1786346358440992 us; error 36 us; skew 500 ppm
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:18.446379 16334 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:18.447190 16334 webserver.cc:460] Webserver started at http://127.0.0.1:40541/ using document root <none> and password file <none>
I20260810 07:19:18.447719 16334 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:18.447783 16334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:18.447962 16334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:18.449846 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "abdb7ad7fee84315bc85209dfa0947f3"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-gjrz"
server_key: "074a537894d93675a340dc79f1ce03bb"
server_key_iv: "b55d7cfdb33a6fb32f4cda791ff2b880"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.450361 16334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "abdb7ad7fee84315bc85209dfa0947f3"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-gjrz"
server_key: "074a537894d93675a340dc79f1ce03bb"
server_key_iv: "b55d7cfdb33a6fb32f4cda791ff2b880"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.453882 16334 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260810 07:19:18.456079 16362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:18.457167 16334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:18.457276 16334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "abdb7ad7fee84315bc85209dfa0947f3"
format_stamp: "Formatted at 2026-08-10 07:19:18 on dist-test-slave-gjrz"
server_key: "074a537894d93675a340dc79f1ce03bb"
server_key_iv: "b55d7cfdb33a6fb32f4cda791ff2b880"
server_key_version: "encryptionkey@0"
I20260810 07:19:18.457360 16334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:18.469187 16334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:18.471570 16334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:18.471738 16334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:18.478030 16334 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33647
I20260810 07:19:18.478052 16446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33647 every 8 connection(s)
I20260810 07:19:18.478848 16334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:18.479038 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16334
I20260810 07:19:18.479158 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:18.479492 12758 external_mini_cluster.cc:1428] Setting key 2d607952bef31c5f896af653dbe42991
I20260810 07:19:18.481884 16448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:18.487936 16448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: Bootstrap starting.
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:19:18.490294 16448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:18.491093 16448 log.cc:824] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:18.493031 16448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: No bootstrap required, opened a new log
I20260810 07:19:18.493345 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.480848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:41863 (local address 127.0.0.1:33647)
0810 07:19:18.481237 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.481255 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.481297 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:19:18.482110 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.482133 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.482533 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.482875 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.483560 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.485021 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.485814 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.486470 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.486629 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.488853 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.488860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.488876 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.488906 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.490630 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.491105 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.491111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.491117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.491245 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.491553 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.491556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.491558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.491978 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.492140 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.492441 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.492635 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":109,"threads_started":1}
I20260810 07:19:18.495908 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:18.496515 16448 raft_consensus.cc:348] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260810 07:19:18.496784 16448 raft_consensus.cc:374] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:18.496846 16448 raft_consensus.cc:729] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abdb7ad7fee84315bc85209dfa0947f3, State: Initialized, Role: FOLLOWER
I20260810 07:19:18.497306 16448 consensus_queue.cc:260] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260810 07:19:18.497469 16448 raft_consensus.cc:388] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:18.497538 16448 raft_consensus.cc:482] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:18.497643 16448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:18.498636 16448 raft_consensus.cc:504] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260810 07:19:18.498996 16448 leader_election.cc:302] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abdb7ad7fee84315bc85209dfa0947f3; no voters:
I20260810 07:19:18.499332 16448 leader_election.cc:288] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:18.499432 16462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:18.499634 16462 raft_consensus.cc:686] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [term 1 LEADER]: Becoming Leader. State: Replica: abdb7ad7fee84315bc85209dfa0947f3, State: Running, Role: LEADER
I20260810 07:19:18.499972 16462 consensus_queue.cc:237] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } }
I20260810 07:19:18.500516 16448 sys_catalog.cc:564] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:18.501037 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abdb7ad7fee84315bc85209dfa0947f3. Latest consensus state: current_term: 1 leader_uuid: "abdb7ad7fee84315bc85209dfa0947f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } } }
I20260810 07:19:18.501156 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:18.501534 16463 sys_catalog.cc:455] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abdb7ad7fee84315bc85209dfa0947f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abdb7ad7fee84315bc85209dfa0947f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33647 } } }
I20260810 07:19:18.501621 16463 sys_catalog.cc:458] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3 [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:18.505352 16490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:18.505430 16490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:18.505489 16471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:18.506297 16471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:18.506510 16453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:18.497995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:38163 (local address 127.0.0.1:33647)
0810 07:19:18.498106 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:18.498111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:18.498336 (+ 225us) server_negotiation.cc:407] Connection header received
0810 07:19:18.498446 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:18.498448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:18.498483 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:18.498539 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:18.499296 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.500001 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.501096 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:18.501425 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:18.501522 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:18.503091 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:18.503094 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:18.503097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:18.503122 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:18.504643 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.505165 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.505169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.505170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.505255 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:18.505630 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:18.505634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:18.505635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:18.505830 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:18.505967 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:18.506284 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 07:19:18.506380 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:19:18.510947 16471 catalog_manager.cc:1347] Generated new cluster ID: 8f0b64eceb8e49c6a5d050b1decbab7f
I20260810 07:19:18.511008 16471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:18.523290 16471 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:18.524175 16471 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:18.527897 16471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abdb7ad7fee84315bc85209dfa0947f3: Generated new TSK 0
I20260810 07:19:18.528427 16471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:18.534399 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16334
W20260810 07:19:18.540890 16310 connection.cc:535] client connection to 127.0.0.1:33647 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33647): Connection reset by peer (error 104)
2026-08-10T07:19:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16502](info): set up 2 sockets
Aug 10 07:19:18 dist-test-slave-gjrz krb5kdc[16502](info): commencing operation
krb5kdc: starting...
W20260810 07:19:19.695027 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:19 dist-test-slave-gjrz krb5kdc[16502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:19.814311 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:19.919215 16524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:19.919459 16524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:19.919516 16524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:19.923404 16524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:19.923455 16524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:19.923480 16524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:19.923501 16524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:19.923525 16524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:19.928520 16524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:19.929405 16524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:19.930311 16524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:19.938020 16524 server_base.cc:1034] running on GCE node
W20260810 07:19:19.937976 16530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:19.938057 16531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:19.937978 16533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:19.938777 16524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:19.939790 16524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:19.940946 16524 hybrid_clock.cc:648] HybridClock initialized: now 1786346359940930 us; error 29 us; skew 500 ppm
Aug 10 07:19:19 dist-test-slave-gjrz krb5kdc[16502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346359, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:19.943519 16524 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:19.944283 16524 webserver.cc:460] Webserver started at http://127.0.0.1:38827/ using document root <none> and password file <none>
I20260810 07:19:19.944761 16524 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:19.944821 16524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:19.945005 16524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:19.946837 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2875e0a8e3f641298ef101703b9c778d"
format_stamp: "Formatted at 2026-08-10 07:19:19 on dist-test-slave-gjrz"
server_key: "b53b7836dd40be9c96640f9db6e9b5d6"
server_key_iv: "6162b27f2745417c02c348664a3b8ca7"
server_key_version: "encryptionkey@0"
I20260810 07:19:19.947378 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2875e0a8e3f641298ef101703b9c778d"
format_stamp: "Formatted at 2026-08-10 07:19:19 on dist-test-slave-gjrz"
server_key: "b53b7836dd40be9c96640f9db6e9b5d6"
server_key_iv: "6162b27f2745417c02c348664a3b8ca7"
server_key_version: "encryptionkey@0"
I20260810 07:19:19.950685 16524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:19.952883 16545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:19.953971 16524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:19.954082 16524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2875e0a8e3f641298ef101703b9c778d"
format_stamp: "Formatted at 2026-08-10 07:19:19 on dist-test-slave-gjrz"
server_key: "b53b7836dd40be9c96640f9db6e9b5d6"
server_key_iv: "6162b27f2745417c02c348664a3b8ca7"
server_key_version: "encryptionkey@0"
I20260810 07:19:19.954165 16524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:19.977679 16524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:19.980945 16524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:19.981140 16524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:19.987931 16524 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38025
I20260810 07:19:19.987932 16624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38025 every 8 connection(s)
I20260810 07:19:19.988790 16524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:19.989344 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16524
I20260810 07:19:19.989477 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:19.989751 12758 external_mini_cluster.cc:1428] Setting key 9f11521cf76a94b6bc4e25b79cc39ffc
I20260810 07:19:19.991709 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:19.997584 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d: Bootstrap starting.
I20260810 07:19:19.998179 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:19.991023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:52377 (local address 127.0.0.1:38025)
0810 07:19:19.991427 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:19.991442 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:19.991474 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:19.992107 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:19.992123 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:19.992441 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:19.992700 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:19.993048 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.993981 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.995178 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:19.995675 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:19.995789 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:19.996600 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:19.996605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:19.996617 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:19.996978 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:19.997028 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:19.997051 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:19.997149 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:19.997330 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 07:19:19.997454 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":114,"threads_started":1}
I20260810 07:19:19.999696 16626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:20.000484 16626 log.cc:824] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:20.000831 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:20.002310 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d: No bootstrap required, opened a new log
I20260810 07:19:20.006407 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:20.002650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:58095 (local address 127.0.0.1:38025)
0810 07:19:20.002754 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:20.002759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:20.002963 (+ 204us) server_negotiation.cc:407] Connection header received
0810 07:19:20.003040 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:20.003043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:20.003084 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:20.003170 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:20.003395 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:20.004131 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:20.004897 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:20.005236 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:20.005333 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:20.005981 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:20.005984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:20.005987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:20.005999 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:20.006025 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:20.006038 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:20.006123 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:20.006219 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:19:20.006250 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:20.006305 16626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } }
I20260810 07:19:20.006589 16626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:20.006651 16626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2875e0a8e3f641298ef101703b9c778d, State: Initialized, Role: FOLLOWER
I20260810 07:19:20.007053 16626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } }
I20260810 07:19:20.007185 16626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:20.007308 16626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:20.007418 16626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:20.008194 16626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } }
I20260810 07:19:20.008472 16626 leader_election.cc:302] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2875e0a8e3f641298ef101703b9c778d; no voters:
I20260810 07:19:20.008714 16626 leader_election.cc:288] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:20.008926 16640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:20.009124 16640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [term 1 LEADER]: Becoming Leader. State: Replica: 2875e0a8e3f641298ef101703b9c778d, State: Running, Role: LEADER
I20260810 07:19:20.009445 16640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } }
I20260810 07:19:20.009744 16626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:20.010792 16642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2875e0a8e3f641298ef101703b9c778d. Latest consensus state: current_term: 1 leader_uuid: "2875e0a8e3f641298ef101703b9c778d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } } }
I20260810 07:19:20.010895 16642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:20.011197 16641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2875e0a8e3f641298ef101703b9c778d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2875e0a8e3f641298ef101703b9c778d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38025 } } }
I20260810 07:19:20.011250 16660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:20.011310 16641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:20.013999 16660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:20.019414 16660 catalog_manager.cc:1347] Generated new cluster ID: 83b55f6fa7eb46d9b5f9697df7878019
I20260810 07:19:20.019470 16660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:20.058239 16660 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:20.059233 16660 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:20.064167 16660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2875e0a8e3f641298ef101703b9c778d: Generated new TSK 0
I20260810 07:19:20.064769 16660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:20.074961 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16524
W20260810 07:19:20.085278 16497 connection.cc:535] client connection to 127.0.0.1:38025 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38025): Connection reset by peer (error 104)
2026-08-10T07:19:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:20 dist-test-slave-gjrz krb5kdc[16680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:20 dist-test-slave-gjrz krb5kdc[16680](info): set up 2 sockets
Aug 10 07:19:20 dist-test-slave-gjrz krb5kdc[16680](info): commencing operation
krb5kdc: starting...
W20260810 07:19:21.241503 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:21 dist-test-slave-gjrz krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:21.354194 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40597
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:21.460034 16700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:21.460278 16700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:21.460319 16700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:21.464264 16700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:21.464313 16700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:21.464330 16700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:21.464345 16700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:21.464361 16700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:21.469389 16700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:21.470288 16700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:21.471189 16700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:21.479041 16700 server_base.cc:1034] running on GCE node
W20260810 07:19:21.478964 16710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:21.478986 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:21.479004 16715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:21.479725 16700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:21.480698 16700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:21.481854 16700 hybrid_clock.cc:648] HybridClock initialized: now 1786346361481844 us; error 25 us; skew 500 ppm
Aug 10 07:19:21 dist-test-slave-gjrz krb5kdc[16680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346361, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:21.484406 16700 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:21.485239 16700 webserver.cc:460] Webserver started at http://127.0.0.1:33233/ using document root <none> and password file <none>
I20260810 07:19:21.485729 16700 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:21.485792 16700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:21.485967 16700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:21.487874 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "388faf293fb14b97be24e874914730f4"
format_stamp: "Formatted at 2026-08-10 07:19:21 on dist-test-slave-gjrz"
server_key: "b87344673a3f9da9418cf6bb61c095ed"
server_key_iv: "77511bcbacdadf950adf823a178734ef"
server_key_version: "encryptionkey@0"
I20260810 07:19:21.488386 16700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "388faf293fb14b97be24e874914730f4"
format_stamp: "Formatted at 2026-08-10 07:19:21 on dist-test-slave-gjrz"
server_key: "b87344673a3f9da9418cf6bb61c095ed"
server_key_iv: "77511bcbacdadf950adf823a178734ef"
server_key_version: "encryptionkey@0"
I20260810 07:19:21.492151 16700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 07:19:21.494589 16729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:21.496109 16700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:21.496233 16700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "388faf293fb14b97be24e874914730f4"
format_stamp: "Formatted at 2026-08-10 07:19:21 on dist-test-slave-gjrz"
server_key: "b87344673a3f9da9418cf6bb61c095ed"
server_key_iv: "77511bcbacdadf950adf823a178734ef"
server_key_version: "encryptionkey@0"
I20260810 07:19:21.496345 16700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:21.500756 16700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:21.503226 16700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:21.503423 16700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:21.509668 16700 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40597
I20260810 07:19:21.509701 16809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40597 every 8 connection(s)
I20260810 07:19:21.510481 16700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:21.513015 16812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:21.517957 16812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4: Bootstrap starting.
I20260810 07:19:21.519147 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16700
I20260810 07:19:21.519300 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:21.519572 12758 external_mini_cluster.cc:1428] Setting key 92596e4d1015b7836ba6dc914beabfc7
I20260810 07:19:21.520470 16812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:21.521238 16812 log.cc:824] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:21.523025 16812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4: No bootstrap required, opened a new log
I20260810 07:19:21.524219 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:21.520629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:49349 (local address 127.0.0.1:40597)
0810 07:19:21.521024 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:21.521038 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:21.521070 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:21.521755 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:21.521780 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:21.522091 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:21.522382 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:21.523089 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:21.523095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:21.523111 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:21.523124 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:21.523183 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:21.523204 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:21.523319 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:21.523457 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 07:19:21.523616 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":110,"threads_started":1}
I20260810 07:19:21.526777 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:21.526710 16812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260810 07:19:21.526944 16812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:21.527006 16812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388faf293fb14b97be24e874914730f4, State: Initialized, Role: FOLLOWER
I20260810 07:19:21.527397 16812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260810 07:19:21.527519 16812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:21.527588 16812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:21.527679 16812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:21.528465 16812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260810 07:19:21.528776 16812 leader_election.cc:302] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 388faf293fb14b97be24e874914730f4; no voters:
I20260810 07:19:21.529053 16812 leader_election.cc:288] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:21.529145 16829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:21.529359 16829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [term 1 LEADER]: Becoming Leader. State: Replica: 388faf293fb14b97be24e874914730f4, State: Running, Role: LEADER
I20260810 07:19:21.529671 16829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } }
I20260810 07:19:21.530198 16812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:21.530503 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:21.528784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:50615 (local address 127.0.0.1:40597)
0810 07:19:21.528892 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:21.528896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:21.529113 (+ 217us) server_negotiation.cc:407] Connection header received
0810 07:19:21.529293 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:21.529296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:21.529331 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:21.529381 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:21.530094 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:21.530097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:21.530100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:21.530113 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:21.530139 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:21.530151 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:21.530240 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:21.530366 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 07:19:21.530404 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:21.530854 16829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 388faf293fb14b97be24e874914730f4. Latest consensus state: current_term: 1 leader_uuid: "388faf293fb14b97be24e874914730f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } } }
I20260810 07:19:21.530951 16829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:21.531824 16832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "388faf293fb14b97be24e874914730f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "388faf293fb14b97be24e874914730f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40597 } } }
I20260810 07:19:21.531932 16832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:21.532532 16841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:21.534871 16841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:21.539543 16841 catalog_manager.cc:1347] Generated new cluster ID: 77a35d19805d41098d6da417e522aae5
I20260810 07:19:21.539604 16841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:21.549417 16841 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:21.549681 16841 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:21.558404 16841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 388faf293fb14b97be24e874914730f4: Generated new TSK 0
I20260810 07:19:21.558943 16841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:21.567518 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16700
2026-08-10T07:19:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1490 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:21 dist-test-slave-gjrz krb5kdc[16868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:21 dist-test-slave-gjrz krb5kdc[16868](info): set up 2 sockets
Aug 10 07:19:21 dist-test-slave-gjrz krb5kdc[16868](info): commencing operation
krb5kdc: starting...
W20260810 07:19:22.724134 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:22 dist-test-slave-gjrz krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:22.835959 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:22.942180 16896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:22.942432 16896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:22.942473 16896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:22.946350 16896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:22.946403 16896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:22.946420 16896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:22.946434 16896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:22.946451 16896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:22.951534 16896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:22.952445 16896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:22.953364 16896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:22.961337 16905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:22.961337 16907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:22.961325 16911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:22.961625 16896 server_base.cc:1034] running on GCE node
I20260810 07:19:22.962159 16896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:22.963198 16896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:22.964361 16896 hybrid_clock.cc:648] HybridClock initialized: now 1786346362964343 us; error 30 us; skew 500 ppm
Aug 10 07:19:22 dist-test-slave-gjrz krb5kdc[16868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346362, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:22.966864 16896 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:22.967751 16896 webserver.cc:460] Webserver started at http://127.0.0.1:38259/ using document root <none> and password file <none>
I20260810 07:19:22.968266 16896 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:22.968330 16896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:22.968515 16896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:22.970468 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "88e5f4d9ed344af390115d7e0846624b"
format_stamp: "Formatted at 2026-08-10 07:19:22 on dist-test-slave-gjrz"
server_key: "e6812c3f6d221ed8042449c0c0611382"
server_key_iv: "8923c96b637435acee9eef70e31f7ac1"
server_key_version: "encryptionkey@0"
I20260810 07:19:22.970993 16896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "88e5f4d9ed344af390115d7e0846624b"
format_stamp: "Formatted at 2026-08-10 07:19:22 on dist-test-slave-gjrz"
server_key: "e6812c3f6d221ed8042449c0c0611382"
server_key_iv: "8923c96b637435acee9eef70e31f7ac1"
server_key_version: "encryptionkey@0"
I20260810 07:19:22.974436 16896 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:19:22.976716 16923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:22.977834 16896 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 07:19:22.977950 16896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "88e5f4d9ed344af390115d7e0846624b"
format_stamp: "Formatted at 2026-08-10 07:19:22 on dist-test-slave-gjrz"
server_key: "e6812c3f6d221ed8042449c0c0611382"
server_key_iv: "8923c96b637435acee9eef70e31f7ac1"
server_key_version: "encryptionkey@0"
I20260810 07:19:22.978065 16896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:23.011596 16896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:23.014122 16896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:23.014287 16896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:23.020859 16896 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37475
I20260810 07:19:23.020875 17004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37475 every 8 connection(s)
I20260810 07:19:23.021739 16896 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:23.024394 17006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:23.029686 17006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: Bootstrap starting.
I20260810 07:19:23.031299 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 16896
I20260810 07:19:23.031432 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:23.031683 12758 external_mini_cluster.cc:1428] Setting key ccab0615470834f22e0e63eaea4b39a8
I20260810 07:19:23.032102 17006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:23.032928 17006 log.cc:824] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:23.035007 17006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: No bootstrap required, opened a new log
I20260810 07:19:23.039141 17006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260810 07:19:23.039381 17006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:23.039434 17006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e5f4d9ed344af390115d7e0846624b, State: Initialized, Role: FOLLOWER
I20260810 07:19:23.039842 17006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260810 07:19:23.039958 17006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:23.040015 17006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:23.040107 17006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 0 FOLLOWER]: Advancing to term 1
Aug 10 07:19:23 dist-test-slave-gjrz krb5kdc[16868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 07:19:23.040993 17006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260810 07:19:23.041355 17006 leader_election.cc:302] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e5f4d9ed344af390115d7e0846624b; no voters:
I20260810 07:19:23.041697 17006 leader_election.cc:288] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:23.041790 17015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:23.042255 17015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [term 1 LEADER]: Becoming Leader. State: Replica: 88e5f4d9ed344af390115d7e0846624b, State: Running, Role: LEADER
I20260810 07:19:23.042553 17015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } }
I20260810 07:19:23.043010 17006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:23.044550 17016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e5f4d9ed344af390115d7e0846624b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } } }
I20260810 07:19:23.044687 17016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:23.045802 17017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e5f4d9ed344af390115d7e0846624b. Latest consensus state: current_term: 1 leader_uuid: "88e5f4d9ed344af390115d7e0846624b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e5f4d9ed344af390115d7e0846624b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37475 } } }
I20260810 07:19:23.045898 17017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:23.046993 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:23.032695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56531 (local address 127.0.0.1:37475)
0810 07:19:23.033136 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:23.033156 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:23.033198 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:19:23.033946 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:23.033969 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:23.034387 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:23.034734 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:23.035423 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:23.036905 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:23.037718 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:23.038456 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:23.038612 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:23.041100 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:23.041107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:23.041123 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:23.041154 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:23.043120 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:23.043756 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:23.043762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:23.043769 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:23.043908 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:23.044306 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:23.044310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:23.044312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:23.044849 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:23.045057 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:23.045838 (+ 781us) server_negotiation.cc:299] Negotiation successful
0810 07:19:23.046059 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
W20260810 07:19:23.048228 17034 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:23.048290 17034 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:23.048347 17033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:23.049026 17033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:23.050490 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:23.054704 17033 catalog_manager.cc:1347] Generated new cluster ID: 6b40d5ed049a4623b6b22d665b3979f8
I20260810 07:19:23.054765 17033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:23.061440 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:23.052775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:56973 (local address 127.0.0.1:37475)
0810 07:19:23.052883 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:23.052888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:23.053013 (+ 125us) server_negotiation.cc:407] Connection header received
0810 07:19:23.053176 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:23.053179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:23.053228 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:23.053313 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:23.054095 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:23.054879 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:23.055998 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:23.056352 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:23.056455 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:23.057941 (+ 1486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:23.057945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:23.057948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:23.057974 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:23.059546 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:23.060180 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:23.060184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:23.060186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:23.060280 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:23.060579 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:23.060582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:23.060584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:23.060787 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:23.060925 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:23.061166 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 07:19:23.061269 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:23.063342 17033 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:23.064180 17033 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:23.077379 17033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e5f4d9ed344af390115d7e0846624b: Generated new TSK 0
I20260810 07:19:23.078014 17033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:23.087906 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 16896
W20260810 07:19:23.109344 16866 connection.cc:535] client connection to 127.0.0.1:37475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37475): Connection reset by peer (error 104)
2026-08-10T07:19:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1533 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:23 dist-test-slave-gjrz krb5kdc[17055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:23 dist-test-slave-gjrz krb5kdc[17055](info): set up 2 sockets
Aug 10 07:19:23 dist-test-slave-gjrz krb5kdc[17055](info): commencing operation
krb5kdc: starting...
W20260810 07:19:24.263969 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:24 dist-test-slave-gjrz krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:24.378867 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:24.486794 17073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:24.487047 17073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:24.487118 17073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:24.491123 17073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:24.491182 17073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:24.491209 17073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:24.491230 17073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:24.491257 17073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:24.496433 17073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33965
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:24.497399 17073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:24.498347 17073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:24.506440 17085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:24.506563 17073 server_base.cc:1034] running on GCE node
W20260810 07:19:24.506500 17079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:24.506474 17080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:24.507323 17073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:24.508358 17073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:24.509522 17073 hybrid_clock.cc:648] HybridClock initialized: now 1786346364509501 us; error 35 us; skew 500 ppm
Aug 10 07:19:24 dist-test-slave-gjrz krb5kdc[17055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346364, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:24.515384 17073 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:24.516213 17073 webserver.cc:460] Webserver started at http://127.0.0.1:46743/ using document root <none> and password file <none>
I20260810 07:19:24.516716 17073 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:24.516777 17073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:24.516943 17073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:24.518868 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "66f30bec30ba4fdaba4e500b585448f5"
format_stamp: "Formatted at 2026-08-10 07:19:24 on dist-test-slave-gjrz"
server_key: "d04d45e557b40a00c6312eacba77a194"
server_key_iv: "f7b6001c5269136a546feed823bfab43"
server_key_version: "encryptionkey@0"
I20260810 07:19:24.519443 17073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "66f30bec30ba4fdaba4e500b585448f5"
format_stamp: "Formatted at 2026-08-10 07:19:24 on dist-test-slave-gjrz"
server_key: "d04d45e557b40a00c6312eacba77a194"
server_key_iv: "f7b6001c5269136a546feed823bfab43"
server_key_version: "encryptionkey@0"
I20260810 07:19:24.522881 17073 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:24.525266 17094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:24.526401 17073 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 07:19:24.526516 17073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "66f30bec30ba4fdaba4e500b585448f5"
format_stamp: "Formatted at 2026-08-10 07:19:24 on dist-test-slave-gjrz"
server_key: "d04d45e557b40a00c6312eacba77a194"
server_key_iv: "f7b6001c5269136a546feed823bfab43"
server_key_version: "encryptionkey@0"
I20260810 07:19:24.526609 17073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:24.552968 17073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:24.555649 17073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:24.555821 17073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:24.562716 17073 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33965
I20260810 07:19:24.562759 17179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33965 every 8 connection(s)
I20260810 07:19:24.563707 17073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:24.564402 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17073
I20260810 07:19:24.564540 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:24.564841 12758 external_mini_cluster.cc:1428] Setting key fa676fcf7d9e202aec1b0486905d8bbe
I20260810 07:19:24.566850 17180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:24.573477 17180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5: Bootstrap starting.
I20260810 07:19:24.574525 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:24.566217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:42319 (local address 127.0.0.1:33965)
0810 07:19:24.566638 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:24.566658 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:24.566701 (+ 43us) server_negotiation.cc:407] Connection header received
0810 07:19:24.567540 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:24.567562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:24.567981 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:24.568335 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:24.568738 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:24.570244 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:24.571084 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:24.571785 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:24.571948 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:24.572727 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:24.572734 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:24.572750 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:24.573210 (+ 460us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:24.573275 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:24.573305 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:24.573434 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:24.573559 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 07:19:24.573696 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260810 07:19:24.575872 17180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:24.576634 17180 log.cc:824] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:24.577405 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:24.578806 17180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5: No bootstrap required, opened a new log
I20260810 07:19:24.583149 17180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } }
I20260810 07:19:24.583431 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:24.579568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:50215 (local address 127.0.0.1:33965)
0810 07:19:24.579677 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:24.579681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:24.579929 (+ 248us) server_negotiation.cc:407] Connection header received
0810 07:19:24.580028 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:24.580029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:24.580068 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:24.580129 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:24.580377 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:24.580863 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:24.581970 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:24.582184 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:24.582250 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:24.583001 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:24.583004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:24.583006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:24.583016 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:24.583040 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:24.583050 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:24.583131 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:24.583287 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 07:19:24.583324 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 07:19:24.583449 17180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:24.583585 17180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66f30bec30ba4fdaba4e500b585448f5, State: Initialized, Role: FOLLOWER
I20260810 07:19:24.584012 17180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } }
I20260810 07:19:24.584139 17180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:24.584196 17180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:24.584292 17180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:24.585119 17180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } }
I20260810 07:19:24.585433 17180 leader_election.cc:302] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66f30bec30ba4fdaba4e500b585448f5; no voters:
I20260810 07:19:24.585711 17180 leader_election.cc:288] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:24.585781 17196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:24.585978 17196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [term 1 LEADER]: Becoming Leader. State: Replica: 66f30bec30ba4fdaba4e500b585448f5, State: Running, Role: LEADER
I20260810 07:19:24.586254 17196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } }
I20260810 07:19:24.586790 17180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:24.587764 17197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66f30bec30ba4fdaba4e500b585448f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } } }
I20260810 07:19:24.587886 17197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:24.588264 17210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:24.588227 17199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66f30bec30ba4fdaba4e500b585448f5. Latest consensus state: current_term: 1 leader_uuid: "66f30bec30ba4fdaba4e500b585448f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66f30bec30ba4fdaba4e500b585448f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33965 } } }
I20260810 07:19:24.588368 17199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:24.590891 17210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:24.596247 17210 catalog_manager.cc:1347] Generated new cluster ID: 32ff7cede0ec42ffb5772e768ec1bcae
I20260810 07:19:24.596306 17210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:24.604302 17210 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:24.605099 17210 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:24.618618 17210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66f30bec30ba4fdaba4e500b585448f5: Generated new TSK 0
I20260810 07:19:24.619235 17210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:24.625219 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17073
W20260810 07:19:24.652269 17053 connection.cc:535] client connection to 127.0.0.1:33965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33965): Connection reset by peer (error 104)
2026-08-10T07:19:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1542 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:24 dist-test-slave-gjrz krb5kdc[17228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:24 dist-test-slave-gjrz krb5kdc[17228](info): set up 2 sockets
Aug 10 07:19:24 dist-test-slave-gjrz krb5kdc[17228](info): commencing operation
krb5kdc: starting...
W20260810 07:19:25.813076 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:25 dist-test-slave-gjrz krb5kdc[17228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 07:19:25.927702 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:26.035995 17246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:26.036227 17246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:26.036283 17246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:26.040218 17246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:26.040271 17246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:26.040295 17246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:26.040318 17246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:26.040344 17246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:26.045452 17246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:26.046397 17246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:26.047391 17246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:26.055325 17257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:26.055346 17260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:26.055446 17256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:26.055603 17246 server_base.cc:1034] running on GCE node
I20260810 07:19:26.056131 17246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:26.057140 17246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:26.058297 17246 hybrid_clock.cc:648] HybridClock initialized: now 1786346366058283 us; error 24 us; skew 500 ppm
Aug 10 07:19:26 dist-test-slave-gjrz krb5kdc[17228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346366, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:26.060894 17246 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 07:19:26.061735 17246 webserver.cc:460] Webserver started at http://127.0.0.1:40471/ using document root <none> and password file <none>
I20260810 07:19:26.062273 17246 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:26.062330 17246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:26.062537 17246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:26.064496 17246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2476b819421d455e84cc1e704736ca81"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-gjrz"
server_key: "ce73c3832ddd286e3c1e96368b496c51"
server_key_iv: "d141f3ba58f4cf49e8fcecc728b82cd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.065014 17246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2476b819421d455e84cc1e704736ca81"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-gjrz"
server_key: "ce73c3832ddd286e3c1e96368b496c51"
server_key_iv: "d141f3ba58f4cf49e8fcecc728b82cd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.068511 17246 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:26.070717 17268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:26.071885 17246 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:26.071998 17246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2476b819421d455e84cc1e704736ca81"
format_stamp: "Formatted at 2026-08-10 07:19:26 on dist-test-slave-gjrz"
server_key: "ce73c3832ddd286e3c1e96368b496c51"
server_key_iv: "d141f3ba58f4cf49e8fcecc728b82cd2"
server_key_version: "encryptionkey@0"
I20260810 07:19:26.072098 17246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:26.076366 17246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:26.078869 17246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:26.079030 17246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:26.085520 17246 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36147
I20260810 07:19:26.085525 17345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36147 every 8 connection(s)
I20260810 07:19:26.086349 17246 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:26.088939 17347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:26.092764 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17246
I20260810 07:19:26.092885 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:26.093151 12758 external_mini_cluster.cc:1428] Setting key e459e9a907f702441634bc1ca163467b
I20260810 07:19:26.094282 17347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81: Bootstrap starting.
I20260810 07:19:26.096438 17347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:26.097277 17347 log.cc:824] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:26.098093 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.094418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:53773 (local address 127.0.0.1:36147)
0810 07:19:26.094809 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:26.094823 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:26.094856 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:26.095502 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:26.095522 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:26.095789 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:26.096064 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:26.096826 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:26.096831 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:26.096842 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:26.096851 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:26.096904 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:26.096922 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:26.097010 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:26.097128 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 07:19:26.097270 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
I20260810 07:19:26.099077 17347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81: No bootstrap required, opened a new log
I20260810 07:19:26.100731 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:26.103329 17347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260810 07:19:26.103595 17347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:26.103655 17347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2476b819421d455e84cc1e704736ca81, State: Initialized, Role: FOLLOWER
I20260810 07:19:26.104091 17347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260810 07:19:26.104243 17347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:26.104318 17347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:26.104427 17347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:26.104439 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:26.102947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.164:51067 (local address 127.0.0.1:36147)
0810 07:19:26.103063 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:26.103066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:26.103191 (+ 125us) server_negotiation.cc:407] Connection header received
0810 07:19:26.103338 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:26.103341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:26.103369 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:26.103429 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:26.104069 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:26.104072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:26.104074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:26.104083 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:26.104106 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:26.104115 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:26.104172 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:26.104275 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 07:19:26.104312 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 07:19:26.105377 17347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260810 07:19:26.105674 17347 leader_election.cc:302] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2476b819421d455e84cc1e704736ca81; no voters:
I20260810 07:19:26.105934 17347 leader_election.cc:288] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:26.106031 17363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:26.106225 17363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [term 1 LEADER]: Becoming Leader. State: Replica: 2476b819421d455e84cc1e704736ca81, State: Running, Role: LEADER
I20260810 07:19:26.106500 17363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } }
I20260810 07:19:26.107012 17347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:26.107975 17367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2476b819421d455e84cc1e704736ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } } }
I20260810 07:19:26.108091 17367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:26.108453 17377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:26.108441 17368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2476b819421d455e84cc1e704736ca81. Latest consensus state: current_term: 1 leader_uuid: "2476b819421d455e84cc1e704736ca81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2476b819421d455e84cc1e704736ca81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36147 } } }
I20260810 07:19:26.109259 17368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:26.111408 17377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:26.116665 17377 catalog_manager.cc:1347] Generated new cluster ID: c55ef37a32824119b13ed18ebf05c0c4
I20260810 07:19:26.116720 17377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:26.125491 17377 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:26.125779 17377 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:26.131115 17377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2476b819421d455e84cc1e704736ca81: Generated new TSK 0
I20260810 07:19:26.131729 17377 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:26.139401 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17246
2026-08-10T07:19:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:26 dist-test-slave-gjrz krb5kdc[17397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:26 dist-test-slave-gjrz krb5kdc[17397](info): set up 2 sockets
Aug 10 07:19:26 dist-test-slave-gjrz krb5kdc[17397](info): commencing operation
krb5kdc: starting...
W20260810 07:19:27.311422 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:27.425535 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39467
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:27.534036 17417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:27.534265 17417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:27.534322 17417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:27.538164 17417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:27.538215 17417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:27.538241 17417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:27.538264 17417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:27.538292 17417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:27.543283 17417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:27.544171 17417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:27.545060 17417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:27.552739 17434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:27.552749 17429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:27.552809 17431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:27.552881 17417 server_base.cc:1034] running on GCE node
I20260810 07:19:27.553514 17417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:27.554482 17417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:27.555624 17417 hybrid_clock.cc:648] HybridClock initialized: now 1786346367555602 us; error 35 us; skew 500 ppm
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:27.558135 17417 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:27.558925 17417 webserver.cc:460] Webserver started at http://127.12.117.190:45311/ using document root <none> and password file <none>
I20260810 07:19:27.559464 17417 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:27.559530 17417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:27.559731 17417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:27.561595 17417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2c8ee888d640495287acdd6038107e77"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-gjrz"
server_key: "77bc56aaecc226a2a1bef6ad2a6a6b45"
server_key_iv: "32c481a8aaa39d7cda643861235b54d2"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.562122 17417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2c8ee888d640495287acdd6038107e77"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-gjrz"
server_key: "77bc56aaecc226a2a1bef6ad2a6a6b45"
server_key_iv: "32c481a8aaa39d7cda643861235b54d2"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.565512 17417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 07:19:27.567652 17442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:27.568836 17417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:27.568948 17417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2c8ee888d640495287acdd6038107e77"
format_stamp: "Formatted at 2026-08-10 07:19:27 on dist-test-slave-gjrz"
server_key: "77bc56aaecc226a2a1bef6ad2a6a6b45"
server_key_iv: "32c481a8aaa39d7cda643861235b54d2"
server_key_version: "encryptionkey@0"
I20260810 07:19:27.569041 17417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:27.591030 17417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:27.593415 17417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:27.593551 17417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:27.599658 17417 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39467
I20260810 07:19:27.599702 17527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39467 every 8 connection(s)
I20260810 07:19:27.600468 17417 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:27.602984 17528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:27.607970 17528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77: Bootstrap starting.
I20260810 07:19:27.610158 17528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:27.610494 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17417
I20260810 07:19:27.610623 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:27.610841 12758 external_mini_cluster.cc:1428] Setting key 5d967c80c6e80c888b94dc870040416f
I20260810 07:19:27.610865 17528 log.cc:824] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:27.612830 17528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77: No bootstrap required, opened a new log
I20260810 07:19:27.616873 17528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } }
I20260810 07:19:27.617117 17528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:27.617173 17528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c8ee888d640495287acdd6038107e77, State: Initialized, Role: FOLLOWER
I20260810 07:19:27.617602 17528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } }
I20260810 07:19:27.617760 17528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:27.617828 17528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:27.617926 17528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:27.618678 17528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } }
I20260810 07:19:27.618999 17528 leader_election.cc:302] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c8ee888d640495287acdd6038107e77; no voters:
I20260810 07:19:27.619346 17528 leader_election.cc:288] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:27.619426 17537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 1 FOLLOWER]: Leader election won for term 1
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:27.619627 17537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [term 1 LEADER]: Becoming Leader. State: Replica: 2c8ee888d640495287acdd6038107e77, State: Running, Role: LEADER
I20260810 07:19:27.619951 17537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } }
I20260810 07:19:27.620553 17528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:27.620950 17539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c8ee888d640495287acdd6038107e77. Latest consensus state: current_term: 1 leader_uuid: "2c8ee888d640495287acdd6038107e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } } }
I20260810 07:19:27.621817 17539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:27.621709 17541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c8ee888d640495287acdd6038107e77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c8ee888d640495287acdd6038107e77" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39467 } } }
I20260810 07:19:27.622557 17541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:27.622975 17549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:27.625281 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:27.611852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50345 (local address 127.12.117.190:39467)
0810 07:19:27.612282 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:27.612301 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:27.612342 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:27.613113 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:27.613137 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:27.613522 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:27.613870 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:27.614478 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.616007 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.616815 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.617475 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.617632 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:27.620207 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:27.620214 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:27.620229 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:27.620259 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:27.622264 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.622876 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.622882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.622888 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.623003 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.623382 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.623385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.623386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.623726 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:27.623887 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:27.624203 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 07:19:27.624424 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":125,"threads_started":1}
I20260810 07:19:27.625759 17549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:27.628670 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:27.631502 17549 catalog_manager.cc:1347] Generated new cluster ID: fd03c0872cb9476cab821419dc4ffff2
I20260810 07:19:27.631561 17549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:27.637858 17536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:27.630762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54969 (local address 127.12.117.190:39467)
0810 07:19:27.630869 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:27.630873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:27.631203 (+ 330us) server_negotiation.cc:407] Connection header received
0810 07:19:27.631352 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:27.631355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:27.631402 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:27.631475 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:27.632173 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.632666 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.633439 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:27.633677 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:27.633764 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:27.634975 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:27.634979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:27.634982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:27.635004 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:27.636258 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.636696 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.636699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.636700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.636772 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:27.637068 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:27.637071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:27.637072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:27.637234 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:27.637370 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:27.637630 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 07:19:27.637715 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:19:27.644011 17549 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:27.645097 17549 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:27.659933 17549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c8ee888d640495287acdd6038107e77: Generated new TSK 0
I20260810 07:19:27.660650 17549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:27.664299 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17417
W20260810 07:19:27.676826 17394 connection.cc:535] client connection to 127.12.117.190:39467 recv error: Network error: failed to read from TLS socket (remote: 127.12.117.190:39467): Connection reset by peer (error 104)
2026-08-10T07:19:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17581](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17581](info): set up 2 sockets
Aug 10 07:19:27 dist-test-slave-gjrz krb5kdc[17581](info): commencing operation
krb5kdc: starting...
W20260810 07:19:28.824713 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:28 dist-test-slave-gjrz krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:28.939774 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34893
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:34893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:29.046909 17615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:29.047156 17615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:29.047212 17615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:29.051127 17615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:29.051187 17615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:29.051213 17615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:29.051234 17615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:29.051275 17615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:29.056313 17615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:34893
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:34893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:29.057217 17615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:29.058131 17615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:29.066208 17622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:29.066277 17625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:29.066295 17627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:29.066623 17615 server_base.cc:1034] running on GCE node
I20260810 07:19:29.067139 17615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:29.068182 17615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:29.069342 17615 hybrid_clock.cc:648] HybridClock initialized: now 1786346369069328 us; error 25 us; skew 500 ppm
Aug 10 07:19:29 dist-test-slave-gjrz krb5kdc[17581](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346369, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:29.071899 17615 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:29.072743 17615 webserver.cc:460] Webserver started at http://127.12.117.190:38779/ using document root <none> and password file <none>
I20260810 07:19:29.073248 17615 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:29.073314 17615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:29.073530 17615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:29.075511 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "2fd725ea24834a05a228e43d0ecfa535"
format_stamp: "Formatted at 2026-08-10 07:19:29 on dist-test-slave-gjrz"
server_key: "84e8666d5800f02a71f8bb348f9b4765"
server_key_iv: "661d2bf4a132f1c429f9ccd5d6455bee"
server_key_version: "encryptionkey@0"
I20260810 07:19:29.076035 17615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "2fd725ea24834a05a228e43d0ecfa535"
format_stamp: "Formatted at 2026-08-10 07:19:29 on dist-test-slave-gjrz"
server_key: "84e8666d5800f02a71f8bb348f9b4765"
server_key_iv: "661d2bf4a132f1c429f9ccd5d6455bee"
server_key_version: "encryptionkey@0"
I20260810 07:19:29.079500 17615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:29.081717 17641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:29.082862 17615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:29.082974 17615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "2fd725ea24834a05a228e43d0ecfa535"
format_stamp: "Formatted at 2026-08-10 07:19:29 on dist-test-slave-gjrz"
server_key: "84e8666d5800f02a71f8bb348f9b4765"
server_key_iv: "661d2bf4a132f1c429f9ccd5d6455bee"
server_key_version: "encryptionkey@0"
I20260810 07:19:29.083070 17615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:29.096055 17615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:29.098609 17615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:29.098778 17615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:29.105381 17615 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:34893
I20260810 07:19:29.105391 17725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:34893 every 8 connection(s)
I20260810 07:19:29.106262 17615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:29.108896 17726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:29.114064 17726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: Bootstrap starting.
I20260810 07:19:29.114866 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17615
I20260810 07:19:29.115037 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:29.115331 12758 external_mini_cluster.cc:1428] Setting key aec24c47722ada005bd2911ea5b16d4f
I20260810 07:19:29.116648 17726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:29.117345 17726 log.cc:824] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:29.119243 17726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: No bootstrap required, opened a new log
I20260810 07:19:29.123179 17726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } }
I20260810 07:19:29.123414 17726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:29.123481 17726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd725ea24834a05a228e43d0ecfa535, State: Initialized, Role: FOLLOWER
I20260810 07:19:29.123872 17726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } }
I20260810 07:19:29.123986 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:29.116363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34003 (local address 127.12.117.190:34893)
0810 07:19:29.116792 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:29.116815 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:29.116850 (+ 35us) server_negotiation.cc:407] Connection header received
0810 07:19:29.117469 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:29.117494 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:29.117901 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:29.118237 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:29.118554 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:29.119577 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:29.120747 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:29.121214 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:29.121327 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:29.122222 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:29.122226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:29.122239 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:29.122594 (+ 355us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:29.122651 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:29.122675 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:29.122783 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:29.122969 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 07:19:29.123105 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":146,"threads_started":1}
I20260810 07:19:29.123996 17726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:29.124109 17726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:29.124212 17726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:29.125181 17726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } }
I20260810 07:19:29.125484 17726 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd725ea24834a05a228e43d0ecfa535; no voters:
I20260810 07:19:29.125793 17726 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:29.125847 17732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:29.126096 17732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd725ea24834a05a228e43d0ecfa535, State: Running, Role: LEADER
I20260810 07:19:29.126466 17732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } }
I20260810 07:19:29.126968 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:29.126992 17726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:29.128181 17736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd725ea24834a05a228e43d0ecfa535. Latest consensus state: current_term: 1 leader_uuid: "2fd725ea24834a05a228e43d0ecfa535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } } }
I20260810 07:19:29.128191 17735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fd725ea24834a05a228e43d0ecfa535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd725ea24834a05a228e43d0ecfa535" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 34893 } } }
I20260810 07:19:29.128317 17736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:29.128317 17735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535 [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:29.131992 17764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:29.132071 17764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:29.132129 17756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:29.132966 17756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:29.133630 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:29.129706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45961 (local address 127.12.117.190:34893)
0810 07:19:29.129801 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:29.129805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:29.130120 (+ 315us) server_negotiation.cc:407] Connection header received
0810 07:19:29.130302 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:29.130305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:29.130351 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:29.130436 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:29.130695 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:29.131211 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:29.132039 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:29.132340 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:29.132433 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:29.133195 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:29.133198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:29.133201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:29.133213 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:29.133239 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:29.133250 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:29.133320 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:29.133472 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 07:19:29.133506 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:19:29.138258 17756 catalog_manager.cc:1347] Generated new cluster ID: 3ea66b9a14bc4f9f80e8782f58053053
I20260810 07:19:29.138324 17756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:29.162385 17756 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:29.163594 17756 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:29.169371 17756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fd725ea24834a05a228e43d0ecfa535: Generated new TSK 0
I20260810 07:19:29.170071 17756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:29.180008 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17615
W20260810 07:19:29.193923 17576 connection.cc:535] client connection to 127.12.117.190:34893 recv error: Network error: failed to read from TLS socket (remote: 127.12.117.190:34893): Connection reset by peer (error 104)
2026-08-10T07:19:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1517 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:29 dist-test-slave-gjrz krb5kdc[17780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:29 dist-test-slave-gjrz krb5kdc[17780](info): set up 2 sockets
Aug 10 07:19:29 dist-test-slave-gjrz krb5kdc[17780](info): commencing operation
krb5kdc: starting...
W20260810 07:19:30.352283 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:30 dist-test-slave-gjrz krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:30.464569 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44163
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:44163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:30.572404 17803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:30.572630 17803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:30.572674 17803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:30.576614 17803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:30.576663 17803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:30.576681 17803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:30.576695 17803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:30.576711 17803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:30.581700 17803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:44163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:44163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:30.582566 17803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:30.583514 17803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:30.591113 17815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.591279 17803 server_base.cc:1034] running on GCE node
W20260810 07:19:30.591104 17819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:30.591084 17817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:30.591967 17803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:30.592904 17803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:30.594048 17803 hybrid_clock.cc:648] HybridClock initialized: now 1786346370594028 us; error 34 us; skew 500 ppm
Aug 10 07:19:30 dist-test-slave-gjrz krb5kdc[17780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346370, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:30.596510 17803 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:30.597294 17803 webserver.cc:460] Webserver started at http://127.12.117.190:34709/ using document root <none> and password file <none>
I20260810 07:19:30.597779 17803 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:30.597841 17803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:30.598016 17803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:30.599962 17803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "43b6997a67494436b99f61d905a3c3d8"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-gjrz"
server_key: "eb60840a66a567ed6214fd7eeba9abb3"
server_key_iv: "691b9df455720abd500deceb2d749593"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.600466 17803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "43b6997a67494436b99f61d905a3c3d8"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-gjrz"
server_key: "eb60840a66a567ed6214fd7eeba9abb3"
server_key_iv: "691b9df455720abd500deceb2d749593"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.603819 17803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 07:19:30.605955 17832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:30.607031 17803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:30.607138 17803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "43b6997a67494436b99f61d905a3c3d8"
format_stamp: "Formatted at 2026-08-10 07:19:30 on dist-test-slave-gjrz"
server_key: "eb60840a66a567ed6214fd7eeba9abb3"
server_key_iv: "691b9df455720abd500deceb2d749593"
server_key_version: "encryptionkey@0"
I20260810 07:19:30.607223 17803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:30.611326 17803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:30.613741 17803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:30.613905 17803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:30.620179 17803 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:44163
I20260810 07:19:30.620195 17912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:44163 every 8 connection(s)
I20260810 07:19:30.620949 17803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:30.623517 17914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:30.628523 17914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8: Bootstrap starting.
I20260810 07:19:30.629557 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17803
I20260810 07:19:30.629693 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:30.629990 12758 external_mini_cluster.cc:1428] Setting key c14aae204c8f4dc7483ed754c1838199
I20260810 07:19:30.631147 17914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:30.631855 17914 log.cc:824] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:30.633621 17914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8: No bootstrap required, opened a new log
I20260810 07:19:30.634356 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.630959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36109 (local address 127.12.117.190:44163)
0810 07:19:30.631376 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.631391 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.631423 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:30.632021 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.632042 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.632301 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.632597 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.633292 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.633296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.633308 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:30.633318 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.633366 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:30.633384 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.633473 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.633599 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.633743 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
I20260810 07:19:30.636942 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:30.637400 17914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } }
I20260810 07:19:30.637605 17914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:30.637660 17914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b6997a67494436b99f61d905a3c3d8, State: Initialized, Role: FOLLOWER
I20260810 07:19:30.638041 17914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } }
I20260810 07:19:30.638195 17914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:30.638260 17914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:30.638355 17914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:30.639279 17914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } }
I20260810 07:19:30.639636 17914 leader_election.cc:302] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43b6997a67494436b99f61d905a3c3d8; no voters:
I20260810 07:19:30.639959 17914 leader_election.cc:288] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:30.640029 17930 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:30.640223 17930 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [term 1 LEADER]: Becoming Leader. State: Replica: 43b6997a67494436b99f61d905a3c3d8, State: Running, Role: LEADER
I20260810 07:19:30.640319 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:30.638809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56235 (local address 127.12.117.190:44163)
0810 07:19:30.638914 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:30.638918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:30.639079 (+ 161us) server_negotiation.cc:407] Connection header received
0810 07:19:30.639153 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:30.639156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:30.639188 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:30.639291 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:30.639830 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:30.639833 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:30.639835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:30.639848 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:30.639874 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:30.639889 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:30.639980 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:30.640142 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 07:19:30.640181 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:30.640638 17930 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } }
I20260810 07:19:30.641182 17914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:30.642275 17932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43b6997a67494436b99f61d905a3c3d8. Latest consensus state: current_term: 1 leader_uuid: "43b6997a67494436b99f61d905a3c3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } } }
I20260810 07:19:30.642395 17932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:30.642653 17931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43b6997a67494436b99f61d905a3c3d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43b6997a67494436b99f61d905a3c3d8" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 44163 } } }
I20260810 07:19:30.642762 17946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:30.642789 17931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:30.645278 17946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:30.649924 17946 catalog_manager.cc:1347] Generated new cluster ID: 1ac2620d3d1042099c010b1fe85754cc
I20260810 07:19:30.649992 17946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:30.662837 17946 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:30.663085 17946 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:30.669801 17946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43b6997a67494436b99f61d905a3c3d8: Generated new TSK 0
I20260810 07:19:30.670344 17946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:30.678985 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17803
2026-08-10T07:19:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:30 dist-test-slave-gjrz krb5kdc[17961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:30 dist-test-slave-gjrz krb5kdc[17961](info): set up 2 sockets
Aug 10 07:19:30 dist-test-slave-gjrz krb5kdc[17961](info): commencing operation
krb5kdc: starting...
W20260810 07:19:31.837210 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:31 dist-test-slave-gjrz krb5kdc[17961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:31.953284 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45013
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:32.060474 17984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:32.060696 17984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:32.060734 17984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:32.064604 17984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:32.064653 17984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:32.064671 17984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:32.064689 17984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:32.064707 17984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:32.069854 17984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:45013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:32.070711 17984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:32.071628 17984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:32.079519 17984 server_base.cc:1034] running on GCE node
W20260810 07:19:32.079533 17996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.079542 17992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:32.079545 17991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:32.080255 17984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:32.081216 17984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:32.082365 17984 hybrid_clock.cc:648] HybridClock initialized: now 1786346372082344 us; error 34 us; skew 500 ppm
Aug 10 07:19:32 dist-test-slave-gjrz krb5kdc[17961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346372, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:32.084857 17984 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:32.085659 17984 webserver.cc:460] Webserver started at http://127.12.117.190:37015/ using document root <none> and password file <none>
I20260810 07:19:32.086153 17984 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:32.086217 17984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:32.086413 17984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:32.088361 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "91f4235f609942f494e19c66ab7938f5"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-gjrz"
server_key: "48df9ad86fc6a26b27934ad042774143"
server_key_iv: "cb444c737d4e5c2e621392e759bb0d9b"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.088925 17984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "91f4235f609942f494e19c66ab7938f5"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-gjrz"
server_key: "48df9ad86fc6a26b27934ad042774143"
server_key_iv: "cb444c737d4e5c2e621392e759bb0d9b"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.092324 17984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 07:19:32.094439 18010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:32.095594 17984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:32.095701 17984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "91f4235f609942f494e19c66ab7938f5"
format_stamp: "Formatted at 2026-08-10 07:19:32 on dist-test-slave-gjrz"
server_key: "48df9ad86fc6a26b27934ad042774143"
server_key_iv: "cb444c737d4e5c2e621392e759bb0d9b"
server_key_version: "encryptionkey@0"
I20260810 07:19:32.095788 17984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:32.104876 17984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:32.107434 17984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:32.107609 17984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:32.114140 17984 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:45013
I20260810 07:19:32.114156 18106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:45013 every 8 connection(s)
I20260810 07:19:32.115000 17984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:32.117604 18110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:32.118146 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 17984
I20260810 07:19:32.118257 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:32.118506 12758 external_mini_cluster.cc:1428] Setting key 62f5b0f245ec88410db960fa685d6b69
I20260810 07:19:32.124145 18110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: Bootstrap starting.
I20260810 07:19:32.126483 18110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:32.127240 18110 log.cc:824] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: Log is configured to *not* fsync() on all Append() calls
Aug 10 07:19:32 dist-test-slave-gjrz krb5kdc[17961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:32.129055 18110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: No bootstrap required, opened a new log
I20260810 07:19:32.132200 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.119800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36951 (local address 127.12.117.190:45013)
0810 07:19:32.120206 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:32.120226 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:32.120267 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:32.121036 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:32.121059 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:32.121490 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:32.121832 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:32.122474 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.123956 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.124808 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.125484 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.125634 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.128077 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:32.128083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:32.128096 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:32.128123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:32.129608 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.130011 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.130015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.130020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.130133 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.130385 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.130388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.130389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.130763 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:32.130942 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:32.131209 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 07:19:32.131501 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":94,"threads_started":1}
I20260810 07:19:32.132417 18110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } }
I20260810 07:19:32.132635 18110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:32.132685 18110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f4235f609942f494e19c66ab7938f5, State: Initialized, Role: FOLLOWER
I20260810 07:19:32.133133 18110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } }
I20260810 07:19:32.133288 18110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:32.133352 18110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:32.133442 18110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:32.134424 18110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } }
I20260810 07:19:32.134769 18110 leader_election.cc:302] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91f4235f609942f494e19c66ab7938f5; no voters:
I20260810 07:19:32.134960 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:32.135121 18110 leader_election.cc:288] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:32.135193 18117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:32.135411 18117 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [term 1 LEADER]: Becoming Leader. State: Replica: 91f4235f609942f494e19c66ab7938f5, State: Running, Role: LEADER
I20260810 07:19:32.135785 18117 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } }
I20260810 07:19:32.136238 18110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:32.137506 18119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91f4235f609942f494e19c66ab7938f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } } }
I20260810 07:19:32.137650 18119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:32.137883 18119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91f4235f609942f494e19c66ab7938f5. Latest consensus state: current_term: 1 leader_uuid: "91f4235f609942f494e19c66ab7938f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91f4235f609942f494e19c66ab7938f5" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 45013 } } }
I20260810 07:19:32.137979 18119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5 [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:32.140946 18152 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:32.141026 18152 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:32.141098 18146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:32.141959 18146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:32.144534 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:32.137201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39449 (local address 127.12.117.190:45013)
0810 07:19:32.137306 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:32.137310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:32.137672 (+ 362us) server_negotiation.cc:407] Connection header received
0810 07:19:32.137862 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:32.137866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:32.137917 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:32.137987 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:32.138798 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.139330 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.140114 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:32.140364 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:32.140421 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:32.141809 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:32.141812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:32.141813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:32.141834 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:32.143049 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.143465 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.143469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.143471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.143543 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:32.143786 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:32.143789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:32.143791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:32.143951 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:32.144060 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:32.144296 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 07:19:32.144400 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:32.148005 18146 catalog_manager.cc:1347] Generated new cluster ID: b1adac28544d44c58f36c54c6fb77bfd
I20260810 07:19:32.148068 18146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:32.171000 18146 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:32.172250 18146 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:32.177500 18146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91f4235f609942f494e19c66ab7938f5: Generated new TSK 0
I20260810 07:19:32.178125 18146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:32.187505 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 17984
2026-08-10T07:19:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:32 dist-test-slave-gjrz krb5kdc[18168](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:32 dist-test-slave-gjrz krb5kdc[18168](info): set up 2 sockets
Aug 10 07:19:32 dist-test-slave-gjrz krb5kdc[18168](info): commencing operation
krb5kdc: starting...
W20260810 07:19:33.358877 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:33 dist-test-slave-gjrz krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:33.472090 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36621
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:33285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:36621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:33.580616 18184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:33.580830 18184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:33.580870 18184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:33.584738 18184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:33.584785 18184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:33.584805 18184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:33.584832 18184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:33.584858 18184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:33.589887 18184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:33285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:36621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:36621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:33.590720 18184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:33.591616 18184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:33.599254 18192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:33.599288 18196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:33.599371 18184 server_base.cc:1034] running on GCE node
W20260810 07:19:33.599292 18194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:33.599985 18184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:33.600916 18184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:33.602058 18184 hybrid_clock.cc:648] HybridClock initialized: now 1786346373602037 us; error 34 us; skew 500 ppm
Aug 10 07:19:33 dist-test-slave-gjrz krb5kdc[18168](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346373, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:33.604489 18184 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:33.605288 18184 webserver.cc:460] Webserver started at http://127.12.117.190:43835/ using document root <none> and password file <none>
I20260810 07:19:33.605772 18184 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:33.605834 18184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:33.606024 18184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:33.607942 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "31def1945bb9427481886d397941d95d"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-gjrz"
server_key: "5b1f7cce3a5dec55998f22f6658982d1"
server_key_iv: "771f8f9d76b5390f9951914e11ee1d09"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.608436 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "31def1945bb9427481886d397941d95d"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-gjrz"
server_key: "5b1f7cce3a5dec55998f22f6658982d1"
server_key_iv: "771f8f9d76b5390f9951914e11ee1d09"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.611783 18184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 07:19:33.613891 18209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:33.614924 18184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:33.615041 18184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "31def1945bb9427481886d397941d95d"
format_stamp: "Formatted at 2026-08-10 07:19:33 on dist-test-slave-gjrz"
server_key: "5b1f7cce3a5dec55998f22f6658982d1"
server_key_iv: "771f8f9d76b5390f9951914e11ee1d09"
server_key_version: "encryptionkey@0"
I20260810 07:19:33.615126 18184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:33.628268 18184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:33.630605 18184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:33.630756 18184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:33.637255 18184 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:36621
I20260810 07:19:33.637265 18297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:36621 every 8 connection(s)
I20260810 07:19:33.638059 18184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:33.640580 18299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:33.645489 18299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: Bootstrap starting.
I20260810 07:19:33.646893 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18184
I20260810 07:19:33.647014 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:33.647255 12758 external_mini_cluster.cc:1428] Setting key 713556e41077c67fb3a508dc4fa3a8fb
I20260810 07:19:33.647791 18299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:33.648615 18299 log.cc:824] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:33.650456 18299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: No bootstrap required, opened a new log
I20260810 07:19:33.653617 18299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } }
I20260810 07:19:33.653877 18299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:33.653947 18299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31def1945bb9427481886d397941d95d, State: Initialized, Role: FOLLOWER
I20260810 07:19:33.654403 18299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } }
I20260810 07:19:33.654548 18299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:33.654619 18299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:33.654714 18299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:33.655650 18299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } }
I20260810 07:19:33.655977 18299 leader_election.cc:302] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31def1945bb9427481886d397941d95d; no voters:
I20260810 07:19:33.656178 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.648244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54727 (local address 127.12.117.190:36621)
0810 07:19:33.648612 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.648628 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.648662 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:19:33.649388 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.649407 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.649713 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.649969 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.650341 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.651346 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.652542 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.653555 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.653673 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.654474 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.654479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.654492 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:33.654929 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.654983 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:33.655006 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.655091 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.655230 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.655448 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
I20260810 07:19:33.656252 18299 leader_election.cc:288] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:33.656359 18306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:33.656596 18306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [term 1 LEADER]: Becoming Leader. State: Replica: 31def1945bb9427481886d397941d95d, State: Running, Role: LEADER
I20260810 07:19:33.656922 18306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } }
I20260810 07:19:33.657426 18299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:33.658051 18312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31def1945bb9427481886d397941d95d. Latest consensus state: current_term: 1 leader_uuid: "31def1945bb9427481886d397941d95d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } } }
I20260810 07:19:33.658720 18312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:33.658617 18311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31def1945bb9427481886d397941d95d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31def1945bb9427481886d397941d95d" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 36621 } } }
I20260810 07:19:33.658990 18311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:33.659112 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 07:19:33.662473 18334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:33.662559 18334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:33.662801 18325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:33.663514 18325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:33.665292 18304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:33.661141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40129 (local address 127.12.117.190:36621)
0810 07:19:33.661235 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:33.661239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:33.661445 (+ 206us) server_negotiation.cc:407] Connection header received
0810 07:19:33.661547 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:33.661550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:33.661592 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:33.661670 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:33.661922 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.662626 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.663740 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:33.664042 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:33.664154 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:33.664902 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:33.664907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:33.664909 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:33.664921 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:33.664949 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:33.664961 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:33.665034 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:33.665124 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 07:19:33.665171 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 07:19:33.668982 18325 catalog_manager.cc:1347] Generated new cluster ID: 68a7e5d5315b482e99adc5437401ce72
I20260810 07:19:33.669046 18325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:33.683696 18325 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:33.684687 18325 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:33.690060 18325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31def1945bb9427481886d397941d95d: Generated new TSK 0
I20260810 07:19:33.690579 18325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:33.695484 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18184
2026-08-10T07:19:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1518 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:33 dist-test-slave-gjrz krb5kdc[18350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:33 dist-test-slave-gjrz krb5kdc[18350](info): set up 2 sockets
Aug 10 07:19:33 dist-test-slave-gjrz krb5kdc[18350](info): commencing operation
krb5kdc: starting...
W20260810 07:19:34.868574 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:34 dist-test-slave-gjrz krb5kdc[18350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:34.985844 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39755
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:32975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:39755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:35.093657 18370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:35.093870 18370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:35.093907 18370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:35.097757 18370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:35.097805 18370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:35.097822 18370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:35.097837 18370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:35.097849 18370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:35.102700 18370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:32975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:39755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:39755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:35.103561 18370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:35.104408 18370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:35.112116 18382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:35.112115 18378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:35.112421 18387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:35.112519 18370 server_base.cc:1034] running on GCE node
I20260810 07:19:35.113020 18370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:35.113978 18370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:35.115131 18370 hybrid_clock.cc:648] HybridClock initialized: now 1786346375115118 us; error 25 us; skew 500 ppm
Aug 10 07:19:35 dist-test-slave-gjrz krb5kdc[18350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346375, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:35.121228 18370 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:35.122007 18370 webserver.cc:460] Webserver started at http://127.12.117.190:46081/ using document root <none> and password file <none>
I20260810 07:19:35.122480 18370 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:35.122540 18370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:35.122699 18370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:35.124595 18370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "8a8d49d7b2be492dbdaa710289fb55ad"
format_stamp: "Formatted at 2026-08-10 07:19:35 on dist-test-slave-gjrz"
server_key: "326f14fcfb03e474f4ff5f6c8c2a88ae"
server_key_iv: "e18b087f6920a2058a7904a270ec42b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:35.125094 18370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "8a8d49d7b2be492dbdaa710289fb55ad"
format_stamp: "Formatted at 2026-08-10 07:19:35 on dist-test-slave-gjrz"
server_key: "326f14fcfb03e474f4ff5f6c8c2a88ae"
server_key_iv: "e18b087f6920a2058a7904a270ec42b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:35.128481 18370 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:35.130651 18398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:35.131778 18370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:35.131886 18370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "8a8d49d7b2be492dbdaa710289fb55ad"
format_stamp: "Formatted at 2026-08-10 07:19:35 on dist-test-slave-gjrz"
server_key: "326f14fcfb03e474f4ff5f6c8c2a88ae"
server_key_iv: "e18b087f6920a2058a7904a270ec42b0"
server_key_version: "encryptionkey@0"
I20260810 07:19:35.131980 18370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:35.136232 18370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:35.138698 18370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:35.138844 18370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:35.145376 18370 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:39755
I20260810 07:19:35.145388 18480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:39755 every 8 connection(s)
I20260810 07:19:35.146221 18370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:35.148834 18481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:35.150939 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18370
I20260810 07:19:35.151064 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:35.151348 12758 external_mini_cluster.cc:1428] Setting key 18453ed6d129ce5eded57546a600a284
I20260810 07:19:35.154289 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad: Bootstrap starting.
I20260810 07:19:35.155973 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.152515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38201 (local address 127.12.117.190:39755)
0810 07:19:35.152886 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.152901 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.152935 (+ 34us) server_negotiation.cc:407] Connection header received
0810 07:19:35.153534 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.153550 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.153820 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.154082 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:35.154842 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:35.154847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:35.154859 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:35.154868 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:35.154917 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:35.154935 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:35.155035 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.155122 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.155309 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260810 07:19:35.156739 18481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:35.157492 18481 log.cc:824] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:35.158656 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:35.159396 18481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad: No bootstrap required, opened a new log
I20260810 07:19:35.162019 18487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:35.160552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38737 (local address 127.12.117.190:39755)
0810 07:19:35.160667 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:35.160671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:35.160806 (+ 135us) server_negotiation.cc:407] Connection header received
0810 07:19:35.160930 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:35.160932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:35.160958 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:35.161011 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:35.161659 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:35.161661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:35.161663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 07:19:35.161673 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:35.161696 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 07:19:35.161705 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:35.161774 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:35.161887 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 07:19:35.161915 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 07:19:35.163489 18481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } }
I20260810 07:19:35.163702 18481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 07:19:35.163756 18481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8d49d7b2be492dbdaa710289fb55ad, State: Initialized, Role: FOLLOWER
I20260810 07:19:35.164165 18481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } }
I20260810 07:19:35.164294 18481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 07:19:35.164356 18481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 07:19:35.164449 18481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:35.165261 18481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } }
I20260810 07:19:35.165567 18481 leader_election.cc:302] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a8d49d7b2be492dbdaa710289fb55ad; no voters:
I20260810 07:19:35.165840 18481 leader_election.cc:288] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 07:19:35.165910 18502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:35.166123 18502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [term 1 LEADER]: Becoming Leader. State: Replica: 8a8d49d7b2be492dbdaa710289fb55ad, State: Running, Role: LEADER
I20260810 07:19:35.166442 18502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } }
I20260810 07:19:35.166944 18481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:35.167955 18504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } } }
I20260810 07:19:35.168090 18504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:35.168408 18509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a8d49d7b2be492dbdaa710289fb55ad. Latest consensus state: current_term: 1 leader_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a8d49d7b2be492dbdaa710289fb55ad" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 39755 } } }
I20260810 07:19:35.168473 18525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:35.168562 18509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:35.171140 18525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:35.175907 18525 catalog_manager.cc:1347] Generated new cluster ID: 563d9821f26241c886dba4989e8af108
I20260810 07:19:35.175967 18525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:35.188793 18525 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:35.189049 18525 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:35.201841 18525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a8d49d7b2be492dbdaa710289fb55ad: Generated new TSK 0
I20260810 07:19:35.202425 18525 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 07:19:35.206456 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18370
2026-08-10T07:19:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1492 ms)
[----------] 18 tests from AuthTokenIssuingTest (27256 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:35 dist-test-slave-gjrz krb5kdc[18543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:35 dist-test-slave-gjrz krb5kdc[18543](info): set up 2 sockets
Aug 10 07:19:35 dist-test-slave-gjrz krb5kdc[18543](info): commencing operation
krb5kdc: starting...
W20260810 07:19:36.371899 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:36.484314 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37521
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:36.589139 18568 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:36.589387 18568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:36.589418 18568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:36.589479 18568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:36.593353 18568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:36.593405 18568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:36.593431 18568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:36.593459 18568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:36.593488 18568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:36.598407 18568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:37521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:36.599287 18568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:36.600167 18568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:36.608050 18577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:36.608108 18579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:36.608111 18568 server_base.cc:1034] running on GCE node
W20260810 07:19:36.608124 18585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:36.608764 18568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:36.609735 18568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:36.610889 18568 hybrid_clock.cc:648] HybridClock initialized: now 1786346376610874 us; error 31 us; skew 500 ppm
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:36.613371 18568 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:36.614151 18568 webserver.cc:460] Webserver started at http://127.12.117.190:38349/ using document root <none> and password file <none>
I20260810 07:19:36.614645 18568 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:36.614708 18568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:36.614897 18568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:36.616814 18568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "98ea33a23d2d46d1898802e13fdbb72b"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "c85d6fd84c75197590b7789b2cf4473e"
server_key_iv: "e936a3890b71fde76175d60123c1910a"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.617314 18568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "98ea33a23d2d46d1898802e13fdbb72b"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "c85d6fd84c75197590b7789b2cf4473e"
server_key_iv: "e936a3890b71fde76175d60123c1910a"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.620743 18568 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 07:19:36.622881 18599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:36.623981 18568 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:36.624090 18568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "98ea33a23d2d46d1898802e13fdbb72b"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "c85d6fd84c75197590b7789b2cf4473e"
server_key_iv: "e936a3890b71fde76175d60123c1910a"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.624179 18568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:36.649317 18568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:36.651836 18568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:36.651997 18568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:36.658203 18568 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:37521
I20260810 07:19:36.658463 18689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:37521 every 8 connection(s)
I20260810 07:19:36.659088 18568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:36.661921 18690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:36.665453 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:36.669121 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18568
I20260810 07:19:36.669226 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:36.669462 12758 external_mini_cluster.cc:1428] Setting key e27745f2665f335fba9d52b106de6d14
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.189@KRBTEST.COM; defaulting to no policy
I20260810 07:19:36.679149 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.667507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.12.117.190:37521)
0810 07:19:36.667924 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:36.667941 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:36.668104 (+ 163us) server_negotiation.cc:407] Connection header received
0810 07:19:36.669078 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:36.669083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:36.669283 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:36.669442 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:36.670249 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.671156 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.672112 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.672477 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.672626 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.674870 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:36.674875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:36.674886 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:36.674906 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:36.676337 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.676765 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.676770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.676771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.676867 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.677142 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.677146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.677147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.677513 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:36.677630 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:36.677972 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:19:36.678188 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"spinlock_wait_cycles":48384,"thread_start_us":155,"threads_started":1}
I20260810 07:19:36.679169 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.667507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37521 (local address 127.0.0.1:50744)
0810 07:19:36.667976 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.668014 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:36.668787 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:36.669469 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:36.669474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:36.669880 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:36.670113 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.670127 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.671296 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.671299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.672002 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.672011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.672613 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.672617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.672711 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.673208 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:36.673223 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:36.674742 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:36.676434 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.676443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.676452 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.676669 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.676951 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.676954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.676957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.677060 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.677626 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:36.677632 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:36.677758 (+ 126us) client_negotiation.cc:769] Sending connection context
0810 07:19:36.677898 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 07:19:36.678274 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":104,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.189@KRBTEST.COM" created.
I20260810 07:19:36.681272 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:36.681977 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.681702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32822)
0810 07:19:36.681820 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.681905 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 07:19:36.682180 18606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
I20260810 07:19:36.682973 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.682786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32836)
0810 07:19:36.682882 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.682924 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 07:19:36.683108 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:36919: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
I20260810 07:19:36.694401 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:36919
--webserver_interface=127.12.117.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:36.732307 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } attempt: 1
I20260810 07:19:36.733022 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.732770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32846)
0810 07:19:36.732895 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.732951 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:19:36.733693 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.733506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32858)
0810 07:19:36.733598 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.733640 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:19:36.733798 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:36919: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
W20260810 07:19:36.800802 18701 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:36.801034 18701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:36.801059 18701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:36.801105 18701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 07:19:36.801971 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } attempt: 2
I20260810 07:19:36.802743 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.802473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32872)
0810 07:19:36.802615 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.802670 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 07:19:36.803386 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.803162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32880)
0810 07:19:36.803280 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.803328 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:19:36.803499 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:36919: Network error: Client connection negotiation failed: client connection to 127.12.117.189:36919: connect: Connection refused (error 111)
W20260810 07:19:36.805315 18701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:36.805361 18701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:36.805382 18701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:36.805411 18701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:36.805439 18701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:36.810426 18701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:36919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.117.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:36.811313 18701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:36.812181 18701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:36.819792 18701 server_base.cc:1034] running on GCE node
W20260810 07:19:36.819731 18710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:36.819711 18707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:36.819727 18708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:36.820441 18701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:36.821388 18701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:36.822536 18701 hybrid_clock.cc:648] HybridClock initialized: now 1786346376822515 us; error 38 us; skew 500 ppm
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:36.825034 18701 init.cc:377] Logged in from keytab as kudu/127.12.117.189@KRBTEST.COM (short username kudu)
I20260810 07:19:36.825795 18701 webserver.cc:460] Webserver started at http://127.12.117.189:39039/ using document root <none> and password file <none>
I20260810 07:19:36.826274 18701 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:36.826331 18701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:36.826536 18701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:36.828454 18701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data/instance:
uuid: "7699784ef8a64df3bf8864f43270c3cd"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "1ba75a48456f60e5b56c60075bdc086e"
server_key_iv: "f0434190e08c1607e21747c2315b31b7"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.828956 18701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal/instance:
uuid: "7699784ef8a64df3bf8864f43270c3cd"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "1ba75a48456f60e5b56c60075bdc086e"
server_key_iv: "f0434190e08c1607e21747c2315b31b7"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.832261 18701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 07:19:36.834384 18730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:36.835464 18701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:36.835562 18701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "7699784ef8a64df3bf8864f43270c3cd"
format_stamp: "Formatted at 2026-08-10 07:19:36 on dist-test-slave-gjrz"
server_key: "1ba75a48456f60e5b56c60075bdc086e"
server_key_iv: "f0434190e08c1607e21747c2315b31b7"
server_key_version: "encryptionkey@0"
I20260810 07:19:36.835675 18701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:36.844774 18701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:36.847458 18701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:36.847617 18701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:36.853726 18701 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.189:36919
I20260810 07:19:36.853971 18808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.189:36919 every 8 connection(s)
I20260810 07:19:36.854550 18701 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:36.857133 18810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:36.860165 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18701
I20260810 07:19:36.860280 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-1/wal/instance
I20260810 07:19:36.860563 12758 external_mini_cluster.cc:1428] Setting key 318d70626f454acf9f464a2d71f62244
I20260810 07:19:36.860867 18810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.117.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.188@KRBTEST.COM" created.
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:36.876034 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.862919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50748 (local address 127.12.117.190:37521)
0810 07:19:36.863031 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:36.863035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:36.863997 (+ 962us) server_negotiation.cc:407] Connection header received
0810 07:19:36.864860 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:36.864864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:36.864928 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:36.864998 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:36.866123 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.866649 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.867770 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.867996 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.868086 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.872222 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:36.872225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:36.872227 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:36.872254 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:36.873815 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.874358 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.874361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.874363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.874471 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.874830 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.874834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.874835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.875086 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:36.875192 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:36.875733 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 07:19:36.875867 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:36.876603 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.863397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37521 (local address 127.0.0.1:50748)
0810 07:19:36.863855 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.863893 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:36.864667 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:36.865171 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:36.865178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:36.865541 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:36.865984 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.865996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.866764 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.866767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.867657 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.867667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.868137 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.868141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.868292 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.868868 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:36.868884 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:36.872044 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:36.873949 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.873954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.873965 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.874232 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.874584 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.874587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.874589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.874725 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.875176 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:36.875181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:36.875467 (+ 286us) client_negotiation.cc:769] Sending connection context
0810 07:19:36.875622 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 07:19:36.875852 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":139,"threads_started":1}
I20260810 07:19:36.877915 18810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
I20260810 07:19:36.886071 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.878355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32896)
0810 07:19:36.878456 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.878469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:36.878537 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:36.879090 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:36.879093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:36.879239 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:36.879342 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.879348 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.880270 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.880273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.880823 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.880833 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.881351 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.881354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.881480 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.881862 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:36.881885 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:36.883012 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:36.884726 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.884730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.884731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.884912 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.885140 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.885143 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.885144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.885223 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.885579 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:36.885593 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:36.885688 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 07:19:36.885799 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 07:19:36.885920 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":31}
I20260810 07:19:36.886238 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.878481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.12.117.189:36919)
0810 07:19:36.878788 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:36.878797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:36.878813 (+ 16us) server_negotiation.cc:407] Connection header received
0810 07:19:36.878843 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:36.878846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:36.878973 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:36.879086 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:36.879437 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.880166 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.880919 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.881254 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.881342 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.883150 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:36.883160 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:36.883173 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:36.883198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:36.884624 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.884995 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.885000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.885001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.885074 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.885296 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.885298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.885299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.885505 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:36.885612 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:36.885942 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 07:19:36.886077 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":225,"thread_start_us":152,"threads_started":1}
I20260810 07:19:36.886731 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:45253
--webserver_interface=127.12.117.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:36.888015 18810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:36.888593 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.888382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60782)
0810 07:19:36.888479 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.888543 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 07:19:36.888757 18735 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
I20260810 07:19:36.889462 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.889282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60792)
0810 07:19:36.889372 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.889416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260810 07:19:36.889616 18810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
I20260810 07:19:36.894655 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } attempt: 3
Aug 10 07:19:36 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:19:36.903205 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.895021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32908)
0810 07:19:36.895139 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.895153 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:36.895226 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:36.895472 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:36.895474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:36.895612 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:36.895695 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.895699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.896659 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.896661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.897565 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:36.897574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.898609 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.898612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:36.898687 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.899078 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:36.899100 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:36.900328 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:36.901958 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.901961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.901962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.902151 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.902416 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:36.902418 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:36.902419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:36.902480 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:36.902811 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:36.902813 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:36.902889 (+ 76us) client_negotiation.cc:769] Sending connection context
0810 07:19:36.902957 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 07:19:36.903051 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:19:36.903281 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.895081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.12.117.189:36919)
0810 07:19:36.895188 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:36.895191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:36.895206 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:19:36.895333 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:36.895337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:36.895383 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:36.895459 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:36.895791 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.896565 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.897660 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:36.898511 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:36.898618 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:36.900454 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:36.900458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:36.900461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:36.900486 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:36.901850 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.902249 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.902252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.902254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.902341 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:36.902546 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:36.902549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:36.902551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:36.902742 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:36.902828 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:36.903015 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 07:19:36.903122 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:36.903859 18690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:36.904428 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.904227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60808)
0810 07:19:36.904323 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.904372 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 07:19:36.905046 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.904841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60822)
0810 07:19:36.904961 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.905001 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 07:19:36.905177 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
I20260810 07:19:36.936324 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 1
I20260810 07:19:36.937018 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.936777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60830)
0810 07:19:36.936897 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.936950 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 07:19:36.937635 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.937439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60844)
0810 07:19:36.937535 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.937578 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:19:36.937777 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
I20260810 07:19:36.938832 18810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 1
I20260810 07:19:36.939404 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.939165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60854)
0810 07:19:36.939289 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.939343 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 07:19:36.939991 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:36.939826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60866)
0810 07:19:36.939907 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:36.939947 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 07:19:36.940099 18810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
W20260810 07:19:36.997426 18818 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:36.997675 18818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:36.997709 18818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:36.997773 18818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:37.001709 18818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:37.001757 18818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:37.001777 18818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:37.001807 18818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:37.001827 18818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:37.006811 18818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:37521,127.12.117.189:36919,127.12.117.188:45253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:45253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.117.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:37.007783 18818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:37.008239 18810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 2
I20260810 07:19:37.008733 18818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:37.008946 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.008691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60868)
0810 07:19:37.008819 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.008877 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:19:37.009575 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.009377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60882)
0810 07:19:37.009471 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.009518 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260810 07:19:37.009680 18810 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
I20260810 07:19:37.012902 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 2
I20260810 07:19:37.013511 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.013281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60884)
0810 07:19:37.013393 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.013444 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 07:19:37.014137 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.013936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60894)
0810 07:19:37.014033 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.014087 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260810 07:19:37.014242 18690 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:45253: Network error: Client connection negotiation failed: client connection to 127.12.117.188:45253: connect: Connection refused (error 111)
W20260810 07:19:37.016765 18830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:37.016748 18835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:37.016752 18828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:37.016989 18818 server_base.cc:1034] running on GCE node
I20260810 07:19:37.017575 18818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:37.018556 18818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:37.019706 18818 hybrid_clock.cc:648] HybridClock initialized: now 1786346377019690 us; error 26 us; skew 500 ppm
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346377, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:37.022265 18818 init.cc:377] Logged in from keytab as kudu/127.12.117.188@KRBTEST.COM (short username kudu)
I20260810 07:19:37.023043 18818 webserver.cc:460] Webserver started at http://127.12.117.188:41677/ using document root <none> and password file <none>
I20260810 07:19:37.023584 18818 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:37.023651 18818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:37.023845 18818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:37.025723 18818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data/instance:
uuid: "d4601fd210ae4a379e26eabcdd74b86e"
format_stamp: "Formatted at 2026-08-10 07:19:37 on dist-test-slave-gjrz"
server_key: "b0aecd5b2732c1dc6832485920551b75"
server_key_iv: "a3b67190244bf39bc4a3b79528ee8885"
server_key_version: "encryptionkey@0"
I20260810 07:19:37.026262 18818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal/instance:
uuid: "d4601fd210ae4a379e26eabcdd74b86e"
format_stamp: "Formatted at 2026-08-10 07:19:37 on dist-test-slave-gjrz"
server_key: "b0aecd5b2732c1dc6832485920551b75"
server_key_iv: "a3b67190244bf39bc4a3b79528ee8885"
server_key_version: "encryptionkey@0"
I20260810 07:19:37.029963 18818 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 07:19:37.032316 18845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:37.033420 18818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 07:19:37.033530 18818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "d4601fd210ae4a379e26eabcdd74b86e"
format_stamp: "Formatted at 2026-08-10 07:19:37 on dist-test-slave-gjrz"
server_key: "b0aecd5b2732c1dc6832485920551b75"
server_key_iv: "a3b67190244bf39bc4a3b79528ee8885"
server_key_version: "encryptionkey@0"
I20260810 07:19:37.033622 18818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:37.047461 18818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:37.050267 18818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:37.050431 18818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:37.057020 18818 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.188:45253
I20260810 07:19:37.057282 18933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.188:45253 every 8 connection(s)
I20260810 07:19:37.057910 18818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:37.060568 18936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:37.062291 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 18818
I20260810 07:19:37.062392 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786346097807571-12758-0/minicluster-data/master-2/wal/instance
I20260810 07:19:37.062641 12758 external_mini_cluster.cc:1428] Setting key 9a84e7710d18ebf6421862730a7f315f
I20260810 07:19:37.064517 18936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346377, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:37.075547 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.066815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50754 (local address 127.12.117.190:37521)
0810 07:19:37.066956 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.066961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.067251 (+ 290us) server_negotiation.cc:407] Connection header received
0810 07:19:37.067401 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.067404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.067452 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.067526 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.068172 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.068650 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.069480 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.069693 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.069778 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.072269 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.072273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.072276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.072302 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.073768 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.074212 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.074215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.074216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.074287 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.074566 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.074570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.074571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.074782 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.074885 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.075253 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.075393 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 07:19:37.076246 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.063914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.12.117.188:45253)
0810 07:19:37.064394 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.064414 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.064455 (+ 41us) server_negotiation.cc:407] Connection header received
0810 07:19:37.065245 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.065270 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.065681 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.066020 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.066785 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.068028 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.068868 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.069318 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.069456 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.071769 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.071776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.071788 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.071813 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.073560 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.073988 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.073993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.073994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.074097 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.074351 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.074355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.074357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.074816 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.074995 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.075200 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.075601 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":346,"spinlock_wait_cycles":2816,"thread_start_us":205,"threads_started":1}
I20260810 07:19:37.076248 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.066777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:37521 (local address 127.0.0.1:50754)
0810 07:19:37.067125 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.067171 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:37.067310 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:37.067594 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:37.067598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:37.067902 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:37.068048 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.068055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.068761 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.068764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.069365 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.069374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.069823 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.069828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.069921 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.070346 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:37.070361 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:37.072135 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:37.073863 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.073872 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.073881 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.074119 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.074371 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.074374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.074375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.074482 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.074872 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:37.074878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:37.075026 (+ 148us) client_negotiation.cc:769] Sending connection context
0810 07:19:37.075201 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 07:19:37.075488 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260810 07:19:37.077334 18936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346377, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:37.085029 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.077740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:36919 (local address 127.0.0.1:32916)
0810 07:19:37.077887 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.077899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:37.077964 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:37.078183 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:37.078186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:37.078342 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:37.078426 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.078433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.079186 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.079189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.079741 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.079752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.080307 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.080312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.080410 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.080797 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:37.080808 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:37.082056 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:37.083714 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.083717 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.083719 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.083893 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.084150 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.084153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.084155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.084230 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.084572 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:37.084575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:37.084667 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 07:19:37.084773 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:19:37.084888 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 07:19:37.085041 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.077845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.12.117.189:36919)
0810 07:19:37.077954 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.077958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.077971 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:37.078050 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.078053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.078096 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.078159 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.078542 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.079084 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.079876 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.080184 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.080281 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.082194 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.082197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.082199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.082221 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.083621 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.083979 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.083983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.083985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.084061 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.084309 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.084312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.084314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.084486 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.084570 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.084827 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.084924 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 07:19:37.085924 18936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:37.086251 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.078279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.12.117.190:37521)
0810 07:19:37.078375 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.078379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.078394 (+ 15us) server_negotiation.cc:407] Connection header received
0810 07:19:37.078514 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.078517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.078555 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.078630 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.079112 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.079655 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.080717 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.080971 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.081044 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.083402 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.083405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.083407 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.083428 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.084629 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.085050 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.085053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.085055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.085143 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.085424 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.085427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.085428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.085640 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.085855 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.085992 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.086114 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346377, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:19:37.093643 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.086484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60908)
0810 07:19:37.086630 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.086641 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:37.086702 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:37.086898 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:37.086900 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:37.087047 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:37.087132 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.087138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.087997 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.088000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.088534 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.088543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.088979 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.088982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.089061 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.089411 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:37.089424 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:37.090688 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:37.092293 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.092296 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.092298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.092498 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.092752 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.092756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.092758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.092835 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.093170 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:37.093173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:37.093263 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 07:19:37.093372 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 07:19:37.093494 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 07:19:37.093732 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.086515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60908 (local address 127.12.117.188:45253)
0810 07:19:37.086631 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.086635 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.086677 (+ 42us) server_negotiation.cc:407] Connection header received
0810 07:19:37.086756 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.086759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.086798 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.086870 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.087237 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.087895 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.088638 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.088885 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.088954 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.090812 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.090815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.090817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.090837 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.092198 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.092589 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.092592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.092593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.092663 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.092919 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.092921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.092922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.093082 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.093153 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.093402 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.093556 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":39}
I20260810 07:19:37.095009 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.087340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.12.117.189:36919)
0810 07:19:37.087443 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.087448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.087461 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:37.087513 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.087515 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.087553 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.087625 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.088147 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.088579 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.089396 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.089598 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.089670 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.092110 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.092113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.092115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.092136 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.093465 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.093903 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.093905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.093907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.093975 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.094260 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.094263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.094264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.094443 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.094657 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.094768 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.094872 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 07:19:37.097313 18936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: Bootstrap starting.
I20260810 07:19:37.097942 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:37.099972 18936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:37.099357 18649 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "98ea33a23d2d46d1898802e13fdbb72b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50758
I20260810 07:19:37.100718 18936 log.cc:824] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:37.100818 18810 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 3
I20260810 07:19:37.102850 18936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: No bootstrap required, opened a new log
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:19:37.106598 18936 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.106793 18936 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4601fd210ae4a379e26eabcdd74b86e, State: Initialized, Role: FOLLOWER
I20260810 07:19:37.107108 18936 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.107548 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.107676 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.108495 18936 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:37.109452 18813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.101188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60920)
0810 07:19:37.101309 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.101323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:37.101400 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:37.101650 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:37.101653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:37.101819 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:37.101921 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.101927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.102783 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.102786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.103622 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.103630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.104144 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.104147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.104227 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.104631 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:37.104644 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:37.105925 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:37.107752 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.107756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.107764 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.107995 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.108311 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.108315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.108316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.108438 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.108914 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:37.108917 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:37.109031 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 07:19:37.109143 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 07:19:37.109288 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:19:37.109469 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.101233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60920 (local address 127.12.117.188:45253)
0810 07:19:37.101362 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.101366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.101379 (+ 13us) server_negotiation.cc:407] Connection header received
0810 07:19:37.101484 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.101487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.101524 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.101598 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.102024 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.102692 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.103728 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.104033 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.104148 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.106055 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.106059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.106061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.106094 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.107631 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.108118 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.108121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.108123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.108199 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.108587 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.108591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.108593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.108805 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.108895 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.109215 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.109331 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 07:19:37.112555 18810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: Bootstrap starting.
I20260810 07:19:37.113377 18690 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } attempt: 3
I20260810 07:19:37.112002 18818 master_runner.cc:186] Error getting master registration for 127.12.117.190:37521: OK, instance_id { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" instance_seqno: 1786346376656524 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:37.115247 18810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:37.115041 18818 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:19:37.114921 18969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:37.115629 18969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:37.116117 18810 log.cc:824] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:37.118381 18810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: No bootstrap required, opened a new log
Aug 10 07:19:37 dist-test-slave-gjrz krb5kdc[18543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346376, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:19:37.121416 18810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.121641 18810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7699784ef8a64df3bf8864f43270c3cd, State: Initialized, Role: FOLLOWER
I20260810 07:19:37.121943 18810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.122364 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.122498 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.123063 18810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:37.125507 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.113764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:45253 (local address 127.0.0.1:60922)
0810 07:19:37.114045 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:37.114059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:37.114135 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:37.115341 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:37.115344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:37.115535 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:37.115640 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.115648 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.116654 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.116658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.117241 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:37.117252 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.118081 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.118086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:37.118183 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.118664 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:37.118678 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:37.120205 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:37.123424 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.123429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.123431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.123702 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.124186 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:37.124205 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:37.124207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:37.124314 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:37.124999 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:37.125003 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:37.125106 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 07:19:37.125193 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 07:19:37.125334 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193}
I20260810 07:19:37.126410 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.114271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60922 (local address 127.12.117.188:45253)
0810 07:19:37.115095 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.115099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.115113 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:37.115145 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.115147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.115187 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.115302 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.115784 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.116516 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.117352 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.117879 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.118420 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.120404 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.120408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.120410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.120461 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.123232 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.123877 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.123881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.123883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.124051 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.124476 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.124479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.124481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.124817 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.125898 (+ 1081us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.126031 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.126208 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754}
W20260810 07:19:37.127784 18986 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:37.127854 18986 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:37.127061 18701 master_runner.cc:186] Error getting master registration for 127.12.117.190:37521: OK, instance_id { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" instance_seqno: 1786346376656524 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:37.130915 18701 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:37.130951 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: Bootstrap starting.
I20260810 07:19:37.133790 18690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:37.134698 18690 log.cc:824] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:37.137028 18690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: No bootstrap required, opened a new log
I20260810 07:19:37.140142 18690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.140378 18690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ea33a23d2d46d1898802e13fdbb72b, State: Initialized, Role: FOLLOWER
I20260810 07:19:37.140722 18690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.141310 18990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.141521 18990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.142108 18690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:37.142400 18649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:19:37.142521 18649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:37.143708 18649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
I20260810 07:19:37.144706 18649 leader_election.cc:288] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [CANDIDATE]: Term 1 election: Requested vote from peers 7699784ef8a64df3bf8864f43270c3cd (127.12.117.189:36919), d4601fd210ae4a379e26eabcdd74b86e (127.12.117.188:45253)
I20260810 07:19:37.145900 18891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98ea33a23d2d46d1898802e13fdbb72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d4601fd210ae4a379e26eabcdd74b86e"
I20260810 07:19:37.146145 18891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:37.146467 18773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98ea33a23d2d46d1898802e13fdbb72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7699784ef8a64df3bf8864f43270c3cd"
I20260810 07:19:37.146683 18773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:37.147507 18891 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea33a23d2d46d1898802e13fdbb72b in term 1.
I20260810 07:19:37.147918 18773 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98ea33a23d2d46d1898802e13fdbb72b in term 1.
I20260810 07:19:37.149948 18606 leader_election.cc:302] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7699784ef8a64df3bf8864f43270c3cd, 98ea33a23d2d46d1898802e13fdbb72b; no voters:
I20260810 07:19:37.151563 18990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:37.151779 18990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [term 1 LEADER]: Becoming Leader. State: Replica: 98ea33a23d2d46d1898802e13fdbb72b, State: Running, Role: LEADER
I20260810 07:19:37.152113 18990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } }
W20260810 07:19:37.153611 18612 tablet.cc:2369] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:37.153955 18990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98ea33a23d2d46d1898802e13fdbb72b. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.154055 18990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:37.154860 18568 master_runner.cc:263] No leader master found from master 98ea33a23d2d46d1898802e13fdbb72b
I20260810 07:19:37.154927 18568 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 07:19:37.158545 19004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:37.158624 19004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:37.158720 19015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:37.159619 19015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:37.177810 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.155948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.12.117.190:37521)
0810 07:19:37.156051 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.156055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.167671 (+ 11616us) server_negotiation.cc:407] Connection header received
0810 07:19:37.167719 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.167722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.167769 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.167844 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.168633 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.169392 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.170540 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.170885 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.170988 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.173166 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.173170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.173173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.173198 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.175718 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.176217 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.176221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.176223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.176309 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.176599 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.176602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.176604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.176811 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.176924 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.177540 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.177652 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 07:19:37.183568 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.155228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.12.117.188:45253)
0810 07:19:37.155354 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.155358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.161834 (+ 6476us) server_negotiation.cc:407] Connection header received
0810 07:19:37.162052 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.162056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.162108 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.162161 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.163225 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.164002 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.165112 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.165461 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.165542 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.167235 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.167239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.167242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.167306 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.174816 (+ 7510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.179255 (+ 4439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.179259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.179287 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.179381 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.182155 (+ 2774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.182158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.182160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.182379 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.182487 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.183241 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.183397 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 07:19:37.185974 18817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:37.154539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32938 (local address 127.12.117.189:36919)
0810 07:19:37.154659 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.154664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.156479 (+ 1815us) server_negotiation.cc:407] Connection header received
0810 07:19:37.156681 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.156685 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.156735 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.156789 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.158460 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.159183 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.160335 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:37.160653 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:37.160731 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:37.179908 (+ 19177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:37.179912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:37.179914 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:37.179953 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:37.181641 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.182809 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.182813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.182815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.182903 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:37.184972 (+ 2069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:37.184975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:37.184977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:37.185190 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:37.185305 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:37.185706 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 07:19:37.185821 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 07:19:37.194223 18891 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [term 1 FOLLOWER]: Refusing update from remote peer 98ea33a23d2d46d1898802e13fdbb72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:37.194913 19011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:37.198819 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98ea33a23d2d46d1898802e13fdbb72b. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.198939 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.201941 18773 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [term 1 FOLLOWER]: Refusing update from remote peer 98ea33a23d2d46d1898802e13fdbb72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:37.202495 19011 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:37.205996 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98ea33a23d2d46d1898802e13fdbb72b. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.206112 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.208115 19011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.208240 19011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:37.211292 18945 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.211403 18945 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.212244 19021 mvcc.cc:205] Tried to move back new op lower bound from 7316874760967524352 to 7316874760818864128. Current Snapshot: MvccSnapshot[applied={T|T < 7316874760967524352}]
I20260810 07:19:37.215709 19015 catalog_manager.cc:1347] Generated new cluster ID: 1d1f938858604d03ae842574d4090680
I20260810 07:19:37.215775 19015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:37.218569 18990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.218678 18990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:37.219679 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98ea33a23d2d46d1898802e13fdbb72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98ea33a23d2d46d1898802e13fdbb72b" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 37521 } } peers { permanent_uuid: "7699784ef8a64df3bf8864f43270c3cd" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 36919 } } peers { permanent_uuid: "d4601fd210ae4a379e26eabcdd74b86e" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 45253 } } }
I20260810 07:19:37.219779 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:37.220199 19027 mvcc.cc:205] Tried to move back new op lower bound from 7316874760967524352 to 7316874760818864128. Current Snapshot: MvccSnapshot[applied={T|T < 7316874760967524352}]
I20260810 07:19:37.229657 19015 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:37.230880 19015 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:37.261664 19015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98ea33a23d2d46d1898802e13fdbb72b: Generated new TSK 0
I20260810 07:19:37.262440 19015 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:19:37.303117 18693 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.299047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.12.117.190:37521)
0810 07:19:37.299170 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.299174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.301218 (+ 2044us) server_negotiation.cc:407] Connection header received
0810 07:19:37.301428 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.301432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.301489 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.301561 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.302990 (+ 1429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50786: BlockingRecv error: recv got EOF from 127.0.0.1:50786 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 07:19:37.304502 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.299946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.12.117.189:36919)
0810 07:19:37.300061 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.300065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.303615 (+ 3550us) server_negotiation.cc:407] Connection header received
0810 07:19:37.303782 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.303785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.303834 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.303882 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.304387 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32946: BlockingRecv error: recv got EOF from 127.0.0.1:32946 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 07:19:37.305951 18941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.300575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.12.117.188:45253)
0810 07:19:37.300683 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.300687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.305002 (+ 4315us) server_negotiation.cc:407] Connection header received
0810 07:19:37.305233 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.305237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.305282 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.305330 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.305804 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60944: BlockingRecv error: recv got EOF from 127.0.0.1:60944 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:19:37.310389 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.308864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.12.117.189:36919)
0810 07:19:37.308969 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.308973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.309414 (+ 441us) server_negotiation.cc:407] Connection header received
0810 07:19:37.309623 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.309626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.309695 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.309746 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.310303 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32954: BlockingRecv error: recv got EOF from 127.0.0.1:32954 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 07:19:37.315342 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.313738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.12.117.189:36919)
0810 07:19:37.313834 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.313838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.314369 (+ 531us) server_negotiation.cc:407] Connection header received
0810 07:19:37.314568 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.314571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.314618 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.314668 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.315179 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32966: BlockingRecv error: recv got EOF from 127.0.0.1:32966 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:19:37.320218 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.319192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.12.117.189:36919)
0810 07:19:37.319311 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.319315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.319525 (+ 210us) server_negotiation.cc:407] Connection header received
0810 07:19:37.319612 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.319615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.319662 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.319747 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.320133 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32974: BlockingRecv error: recv got EOF from 127.0.0.1:32974 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 07:19:37.327225 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.326116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32982 (local address 127.12.117.189:36919)
0810 07:19:37.326242 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.326246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.326460 (+ 214us) server_negotiation.cc:407] Connection header received
0810 07:19:37.326582 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.326585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.326628 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.326704 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.327142 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32982: BlockingRecv error: recv got EOF from 127.0.0.1:32982 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 07:19:37.337868 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.336938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.12.117.189:36919)
0810 07:19:37.337047 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.337050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.337184 (+ 134us) server_negotiation.cc:407] Connection header received
0810 07:19:37.337298 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.337300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.337336 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.337396 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.337783 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32998: BlockingRecv error: recv got EOF from 127.0.0.1:32998 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:19:37.356381 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.355487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.12.117.189:36919)
0810 07:19:37.355592 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.355595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.355791 (+ 196us) server_negotiation.cc:407] Connection header received
0810 07:19:37.355845 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.355847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.355890 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.355963 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.356304 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33004: BlockingRecv error: recv got EOF from 127.0.0.1:33004 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:19:37.390869 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.389992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.12.117.189:36919)
0810 07:19:37.390154 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.390157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.390241 (+ 84us) server_negotiation.cc:407] Connection header received
0810 07:19:37.390337 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.390339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.390379 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.390442 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.390771 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33016: BlockingRecv error: recv got EOF from 127.0.0.1:33016 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 07:19:37.457775 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.456695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.12.117.189:36919)
0810 07:19:37.456823 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.456827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.456963 (+ 136us) server_negotiation.cc:407] Connection header received
0810 07:19:37.457112 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.457115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.457159 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.457226 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.457665 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33030: BlockingRecv error: recv got EOF from 127.0.0.1:33030 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 07:19:37.588924 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.587772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.12.117.189:36919)
0810 07:19:37.587914 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.587919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.588046 (+ 127us) server_negotiation.cc:407] Connection header received
0810 07:19:37.588203 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.588206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.588254 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.588323 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.588796 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33044: BlockingRecv error: recv got EOF from 127.0.0.1:33044 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:19:37.847733 18817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:37.846719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.12.117.189:36919)
0810 07:19:37.846844 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:37.846849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:37.846989 (+ 140us) server_negotiation.cc:407] Connection header received
0810 07:19:37.847116 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:37.847118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:37.847162 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:37.847227 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:37.847616 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33052: BlockingRecv error: recv got EOF from 127.0.0.1:33052 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 07:19:38.116914 18969 catalog_manager.cc:1259] Loaded cluster ID: 1d1f938858604d03ae842574d4090680
I20260810 07:19:38.116994 18969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: loading cluster ID for follower catalog manager: success
I20260810 07:19:38.118698 18969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: acquiring CA information for follower catalog manager: success
I20260810 07:19:38.119874 18969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d4601fd210ae4a379e26eabcdd74b86e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:38.128908 18986 catalog_manager.cc:1259] Loaded cluster ID: 1d1f938858604d03ae842574d4090680
I20260810 07:19:38.128960 18986 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: loading cluster ID for follower catalog manager: success
I20260810 07:19:38.130424 18986 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: acquiring CA information for follower catalog manager: success
I20260810 07:19:38.131426 18986 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7699784ef8a64df3bf8864f43270c3cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:38.365643 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.361674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.12.117.189:36919)
0810 07:19:38.361905 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.361910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.361979 (+ 69us) server_negotiation.cc:407] Connection header received
0810 07:19:38.362141 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.362144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.362265 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.362366 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:38.362638 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.363171 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.364032 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.364266 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.364356 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.364702 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:38.365256 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:38.365372 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.365464 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.365520 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":61,"threads_started":1}
I20260810 07:19:38.371711 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.368394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.12.117.188:45253)
0810 07:19:38.368616 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.368621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.368635 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:38.368721 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.368723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.368824 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.368921 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:38.369120 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.369621 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.370403 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.370628 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.370711 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.370971 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:38.371354 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:38.371450 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.371517 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.371568 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":64,"threads_started":1}
I20260810 07:19:38.377943 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.374133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.12.117.189:36919)
0810 07:19:38.374233 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.374236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.374463 (+ 227us) server_negotiation.cc:407] Connection header received
0810 07:19:38.374643 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.374647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.374692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.374729 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:38.375019 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.375638 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.376788 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.377125 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.377199 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.377557 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:38.377624 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:38.377682 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.377777 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.377822 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:38.378469 19164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.374143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.12.117.190:37521)
0810 07:19:38.374359 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.374363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.374440 (+ 77us) server_negotiation.cc:407] Connection header received
0810 07:19:38.374650 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.374654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.374806 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.374926 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:38.375183 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.375808 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.376820 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.377175 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.377268 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.377636 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:38.377919 (+ 283us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:38.377998 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.378137 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.378200 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260810 07:19:38.378584 19149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:38.374541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.12.117.188:45253)
0810 07:19:38.374651 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:38.374655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:38.374912 (+ 257us) server_negotiation.cc:407] Connection header received
0810 07:19:38.375046 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:38.375050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:38.375097 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:38.375134 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:38.375492 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.376208 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.377290 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:38.377593 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:38.377663 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:38.378193 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:38.378265 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:38.378339 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:38.378438 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 07:19:38.378482 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 07:19:38.381247 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18568
I20260810 07:19:38.411820 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18701
I20260810 07:19:38.418668 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 18818
2026-08-10T07:19:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3217 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 07:19:38 dist-test-slave-gjrz krb5kdc[19170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 07:19:38 dist-test-slave-gjrz krb5kdc[19170](info): set up 2 sockets
Aug 10 07:19:38 dist-test-slave-gjrz krb5kdc[19170](info): commencing operation
krb5kdc: starting...
W20260810 07:19:39.579890 12758 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/test-user.keytab.
Aug 10 07:19:39 dist-test-slave-gjrz krb5kdc[19170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T07:19:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T07:19:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.117.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
Entry for principal kudu/127.12.117.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab.
I20260810 07:19:39.691927 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:33233
--webserver_interface=127.12.117.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 07:19:39.800256 19187 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:39.800526 19187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:39.800573 19187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:39.800642 19187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:39.805671 19187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:39.805737 19187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:39.805763 19187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:39.805784 19187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:39.805810 19187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:39.811563 19187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.190.keytab
--principal=kudu/127.12.117.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.190:33233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.117.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:39.812677 19187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:39.813599 19187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:39.823115 19193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.823122 19194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:39.823155 19196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:39.823899 19187 server_base.cc:1034] running on GCE node
I20260810 07:19:39.824476 19187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:39.825474 19187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:39.826635 19187 hybrid_clock.cc:648] HybridClock initialized: now 1786346379826619 us; error 27 us; skew 500 ppm
Aug 10 07:19:39 dist-test-slave-gjrz krb5kdc[19170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:39.829905 19187 init.cc:377] Logged in from keytab as kudu/127.12.117.190@KRBTEST.COM (short username kudu)
I20260810 07:19:39.830734 19187 webserver.cc:460] Webserver started at http://127.12.117.190:43041/ using document root <none> and password file <none>
I20260810 07:19:39.831244 19187 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:39.831341 19187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:39.831534 19187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:39.833438 19187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data/instance:
uuid: "00508c8e2cca4cd68a18b782290b334e"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-gjrz"
server_key: "01e4e2b350f21e8ee7148a0f71206bfb"
server_key_iv: "eacc2acef0829aae12ab7c3e0cd946f3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.833962 19187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance:
uuid: "00508c8e2cca4cd68a18b782290b334e"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-gjrz"
server_key: "01e4e2b350f21e8ee7148a0f71206bfb"
server_key_iv: "eacc2acef0829aae12ab7c3e0cd946f3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.838024 19187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 07:19:39.840929 19206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:39.842327 19187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 07:19:39.842489 19187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal
uuid: "00508c8e2cca4cd68a18b782290b334e"
format_stamp: "Formatted at 2026-08-10 07:19:39 on dist-test-slave-gjrz"
server_key: "01e4e2b350f21e8ee7148a0f71206bfb"
server_key_iv: "eacc2acef0829aae12ab7c3e0cd946f3"
server_key_version: "encryptionkey@0"
I20260810 07:19:39.842674 19187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:39.860642 19187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:39.863001 19187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:39.863153 19187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:39.869377 19187 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.190:33233
I20260810 07:19:39.869626 19298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.190:33233 every 8 connection(s)
I20260810 07:19:39.870210 19187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/data/info.pb
I20260810 07:19:39.872725 19303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:39.876118 19303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:39.877151 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19187
I20260810 07:19:39.877262 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-0/wal/instance
I20260810 07:19:39.877509 12758 external_mini_cluster.cc:1428] Setting key 2bcec8997ad834a4cd3ea0255b0a41d1
Aug 10 07:19:39 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.117.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.189@KRBTEST.COM" created.
I20260810 07:19:39.890677 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.878143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.12.117.190:33233)
0810 07:19:39.878978 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:39.878987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:39.879019 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:39.880015 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:39.880022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:39.880277 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:39.880476 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:39.881241 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.882140 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.883582 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.883966 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.884089 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.886554 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:39.886560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:39.886570 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:39.886592 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:39.888101 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.888577 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.888583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.888584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.888685 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:39.888982 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:39.888986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:39.888987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:39.889365 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:39.889502 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:39.889860 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 07:19:39.890048 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":743,"spinlock_wait_cycles":48512,"thread_start_us":145,"threads_started":1}
I20260810 07:19:39.890676 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.878143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33233 (local address 127.0.0.1:56034)
0810 07:19:39.878645 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.878696 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:39.879622 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:39.880463 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:39.880467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:39.880771 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:39.881080 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.881099 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.882286 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.882290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.883463 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:39.883476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:39.884114 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:39.884118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:39.884276 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:39.884767 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:39.884786 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:39.886384 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:39.888207 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.888221 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.888228 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.888450 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.888772 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:39.888775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:39.888777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:39.888880 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:39.889482 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:39.889487 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:39.889615 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 07:19:39.889748 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 07:19:39.889974 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":37,"thread_start_us":133,"threads_started":1}
I20260810 07:19:39.893101 19303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:39.893716 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.893498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50956)
0810 07:19:39.893601 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.893666 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 07:19:39.893913 19211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
I20260810 07:19:39.894681 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.894491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50972)
0810 07:19:39.894593 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.894634 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 07:19:39.894816 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:33355: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
Entry for principal kudu/127.12.117.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab.
I20260810 07:19:39.903838 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:33355
--webserver_interface=127.12.117.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:39.944033 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } attempt: 1
I20260810 07:19:39.944793 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.944516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50974)
0810 07:19:39.944658 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.944718 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 07:19:39.945590 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:39.945337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50978)
0810 07:19:39.945473 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:39.945527 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 07:19:39.945730 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:33355: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
W20260810 07:19:40.009317 19316 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:40.009552 19316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:40.009575 19316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:40.009632 19316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 07:19:40.013494 19316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:40.013540 19316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:40.013559 19316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:40.013589 19316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:40.013615 19316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:40.013880 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } attempt: 2
I20260810 07:19:40.014639 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.014375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50982)
0810 07:19:40.014507 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.014565 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 07:19:40.015312 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.015098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:50996)
0810 07:19:40.015191 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.015233 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 07:19:40.015426 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.189:33355: Network error: Client connection negotiation failed: client connection to 127.12.117.189:33355: connect: Connection refused (error 111)
I20260810 07:19:40.018950 19316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.189.keytab
--principal=kudu/127.12.117.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.189:33355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.117.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:40.019851 19316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:40.020716 19316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 07:19:40.028330 19321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.028367 19322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.028339 19326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:40.028549 19316 server_base.cc:1034] running on GCE node
I20260810 07:19:40.029145 19316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:40.030098 19316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:40.031255 19316 hybrid_clock.cc:648] HybridClock initialized: now 1786346380031236 us; error 27 us; skew 500 ppm
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:40.033766 19316 init.cc:377] Logged in from keytab as kudu/127.12.117.189@KRBTEST.COM (short username kudu)
I20260810 07:19:40.034539 19316 webserver.cc:460] Webserver started at http://127.12.117.189:44895/ using document root <none> and password file <none>
I20260810 07:19:40.035036 19316 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:40.035099 19316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:40.035317 19316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:40.037199 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data/instance:
uuid: "7e0b72181bfa4ced911e2bbd741a49d1"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "e0b0798c554136689275174d6bc87660"
server_key_iv: "0e1f711dcc68d8935ef00977a4d561f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.037719 19316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal/instance:
uuid: "7e0b72181bfa4ced911e2bbd741a49d1"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "e0b0798c554136689275174d6bc87660"
server_key_iv: "0e1f711dcc68d8935ef00977a4d561f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.041160 19316 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 07:19:40.043325 19338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:40.044399 19316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 07:19:40.044507 19316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal
uuid: "7e0b72181bfa4ced911e2bbd741a49d1"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "e0b0798c554136689275174d6bc87660"
server_key_iv: "0e1f711dcc68d8935ef00977a4d561f6"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.044598 19316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:40.074719 19316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:40.077193 19316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:40.077353 19316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:40.083597 19316 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.189:33355
I20260810 07:19:40.083832 19433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.189:33355 every 8 connection(s)
I20260810 07:19:40.084438 19316 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/data/info.pb
I20260810 07:19:40.086969 19434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:40.089273 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19316
I20260810 07:19:40.089391 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-1/wal/instance
I20260810 07:19:40.089655 12758 external_mini_cluster.cc:1428] Setting key ca9a53a67f6b1c42b85f3d6741e25c4a
I20260810 07:19:40.090467 19434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.117.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.117.188@KRBTEST.COM" created.
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:40.105160 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.092185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.12.117.190:33233)
0810 07:19:40.092320 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.092324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.093091 (+ 767us) server_negotiation.cc:407] Connection header received
0810 07:19:40.093965 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.093969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.094028 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.094096 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.095150 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.095707 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.096776 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.097008 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.097150 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.100965 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.100970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.100973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.100998 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.103013 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.103654 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.103657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.103659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.103754 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.104097 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.104099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.104101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.104261 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.104330 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.104934 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.105041 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 07:19:40.106132 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.092486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33233 (local address 127.0.0.1:56038)
0810 07:19:40.092937 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.092985 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.093767 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.094267 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.094273 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.094592 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.095004 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.095016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.095824 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.095827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.096663 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.096672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.097182 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.097186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.097364 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.097808 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.097823 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.100774 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.103135 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.103141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.103154 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.103556 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.103841 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.103844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.103846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.104016 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.104360 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.104367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.104678 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.104886 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.105196 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
I20260810 07:19:40.106568 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } attempt: 3
I20260810 07:19:40.107514 19434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
Entry for principal kudu/127.12.117.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab.
I20260810 07:19:40.115736 12758 external_mini_cluster.cc:1326] Running /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
/tmp/dist-test-task3eVEGZ/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:35285
--webserver_interface=127.12.117.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.117.148:39601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 07:19:40.116464 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.106947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:51012)
0810 07:19:40.107111 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.107123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.107193 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.108121 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.108124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.108293 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.108409 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.108416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.109877 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.109881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.110454 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.110465 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.111050 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.111054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.111167 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.111613 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.111624 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.112847 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.114992 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.114995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.114997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.115214 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.115519 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.115522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.115525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.115615 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.116008 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.116011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.116106 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.116191 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.116318 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 07:19:40.116567 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.107427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.12.117.189:33355)
0810 07:19:40.107718 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.107729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.107750 (+ 21us) server_negotiation.cc:407] Connection header received
0810 07:19:40.107790 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.107795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.107966 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.108504 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.108630 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.109756 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.110556 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.110944 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.111037 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.112959 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.112969 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.112982 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.113008 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.114863 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.115343 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.115347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.115348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.115425 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.115696 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.115699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.115701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.115910 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.116023 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.116282 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.116409 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":204,"thread_start_us":121,"threads_started":1}
I20260810 07:19:40.117413 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.107946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:51018)
0810 07:19:40.108093 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.108107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.108175 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.108461 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.108463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.108578 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.108713 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.108719 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.109860 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.109863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.110396 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.110404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.111059 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.111062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.111173 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.111713 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.111726 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.113331 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.116072 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.116074 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.116076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.116282 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.116524 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.116526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.116528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.116608 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.117002 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.117005 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.117082 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.117170 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.117274 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":20}
I20260810 07:19:40.117879 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.107979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.12.117.189:33355)
0810 07:19:40.108271 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.108280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.108291 (+ 11us) server_negotiation.cc:407] Connection header received
0810 07:19:40.108322 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.108324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.108360 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.108494 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.108809 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.109718 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.110493 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.110958 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.111053 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.113471 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.113474 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.113476 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.113499 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.115970 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.116371 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.116373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.116375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.116448 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.116686 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.116689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.116691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.116883 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.116968 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.117682 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.117765 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":227,"server-negotiator.queue_time_us":235,"spinlock_wait_cycles":1792,"thread_start_us":84,"threads_started":1}
I20260810 07:19:40.118251 19303 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:40.118682 19434 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } has no permanent_uuid. Determining permanent_uuid...
I20260810 07:19:40.118846 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.118643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57168)
0810 07:19:40.118731 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.118782 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 07:19:40.119383 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.119094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57182)
0810 07:19:40.119208 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.119321 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:19:40.119570 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.119306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57194)
0810 07:19:40.119465 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.119510 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260810 07:19:40.119601 19343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
W20260810 07:19:40.120024 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
I20260810 07:19:40.120390 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.120183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57208)
0810 07:19:40.120281 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.120332 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 07:19:40.120543 19434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
I20260810 07:19:40.151180 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 1
I20260810 07:19:40.151916 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.151656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57224)
0810 07:19:40.151783 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.151841 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 07:19:40.152609 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.152417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57240)
0810 07:19:40.152513 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.152555 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 07:19:40.152719 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
I20260810 07:19:40.169771 19434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 1
I20260810 07:19:40.170461 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.170166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57254)
0810 07:19:40.170302 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.170367 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 07:19:40.171123 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.170915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57264)
0810 07:19:40.171006 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.171059 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260810 07:19:40.171248 19434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
W20260810 07:19:40.224848 19445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 07:19:40.225077 19445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 07:19:40.225103 19445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 07:19:40.225160 19445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 07:19:40.227847 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 2
I20260810 07:19:40.228564 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.228311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57272)
0810 07:19:40.228439 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.228496 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 07:19:40.229135 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.228967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57282)
0810 07:19:40.229044 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.229085 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 07:19:40.229245 19303 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
W20260810 07:19:40.229262 19445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 07:19:40.229310 19445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 07:19:40.229346 19445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 07:19:40.229377 19445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 07:19:40.229404 19445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 07:19:40.234472 19445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.117.148:39601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.117.190:33233,127.12.117.189:33355,127.12.117.188:35285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/krb5kdc/kudu_127.12.117.188.keytab
--principal=kudu/127.12.117.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.117.188:35285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.117.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 06:33:02 UTC on ccycloud.quasar-daqefu.root.comops.site
build id 4577113
I20260810 07:19:40.235368 19445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 07:19:40.236255 19445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 07:19:40.239408 19434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 2
I20260810 07:19:40.240088 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.239803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57296)
0810 07:19:40.239940 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.240006 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 07:19:40.240850 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.240622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57304)
0810 07:19:40.240731 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.240784 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 07:19:40.240990 19434 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.117.188:35285: Network error: Client connection negotiation failed: client connection to 127.12.117.188:35285: connect: Connection refused (error 111)
W20260810 07:19:40.244285 19461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.244274 19456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 07:19:40.244273 19457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 07:19:40.244422 19445 server_base.cc:1034] running on GCE node
I20260810 07:19:40.245067 19445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 07:19:40.246080 19445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 07:19:40.247251 19445 hybrid_clock.cc:648] HybridClock initialized: now 1786346380247237 us; error 27 us; skew 500 ppm
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 07:19:40.257066 19445 init.cc:377] Logged in from keytab as kudu/127.12.117.188@KRBTEST.COM (short username kudu)
I20260810 07:19:40.257862 19445 webserver.cc:460] Webserver started at http://127.12.117.188:35905/ using document root <none> and password file <none>
I20260810 07:19:40.258375 19445 fs_manager.cc:359] Metadata directory not provided
I20260810 07:19:40.258437 19445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 07:19:40.258597 19445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 07:19:40.260491 19445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data/instance:
uuid: "0b1c64fca2654386a598c6491232de97"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "3b5e6b39d3026aabf5fb0fc1462e3d26"
server_key_iv: "615a14a52b38436118692944eea46802"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.261054 19445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal/instance:
uuid: "0b1c64fca2654386a598c6491232de97"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "3b5e6b39d3026aabf5fb0fc1462e3d26"
server_key_iv: "615a14a52b38436118692944eea46802"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.264494 19445 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 07:19:40.266685 19479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 07:19:40.267803 19445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 07:19:40.267905 19445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data,/tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal
uuid: "0b1c64fca2654386a598c6491232de97"
format_stamp: "Formatted at 2026-08-10 07:19:40 on dist-test-slave-gjrz"
server_key: "3b5e6b39d3026aabf5fb0fc1462e3d26"
server_key_iv: "615a14a52b38436118692944eea46802"
server_key_version: "encryptionkey@0"
I20260810 07:19:40.268028 19445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 07:19:40.278378 19445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 07:19:40.280773 19445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 07:19:40.280928 19445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 07:19:40.287317 19445 rpc_server.cc:307] RPC server started. Bound to: 127.12.117.188:35285
I20260810 07:19:40.287537 19568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.117.188:35285 every 8 connection(s)
I20260810 07:19:40.288180 19445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/data/info.pb
I20260810 07:19:40.290632 19569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 07:19:40.291663 12758 external_mini_cluster.cc:1388] Started /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu as pid 19445
I20260810 07:19:40.291786 12758 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task3eVEGZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786346097807571-12758-0/minicluster-data/master-2/wal/instance
I20260810 07:19:40.292100 12758 external_mini_cluster.cc:1428] Setting key 11744113f9284081dfd125eb6c04170c
I20260810 07:19:40.294507 19569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:40.305191 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.296298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.12.117.190:33233)
0810 07:19:40.296400 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.296403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.296731 (+ 328us) server_negotiation.cc:407] Connection header received
0810 07:19:40.296842 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.296846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.296894 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.296954 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.297676 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.298181 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.299140 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.299460 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.299540 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.301842 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.301847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.301849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.301888 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.303194 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.303685 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.303689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.303690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.303765 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.304067 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.304069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.304071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.304310 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.304407 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.304848 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.305026 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:40.305784 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.293384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.12.117.188:35285)
0810 07:19:40.293755 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.293769 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.293802 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:40.294395 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.294410 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.294729 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.295010 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.295875 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.297166 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.298118 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.298642 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.298753 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.301389 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.301395 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.301404 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.301425 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.303137 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.303618 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.303622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.303623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.303750 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.304037 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.304041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.304042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.304464 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.304637 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.304843 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.305032 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":103,"threads_started":1}
I20260810 07:19:40.306222 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.296243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.190:33233 (local address 127.0.0.1:56040)
0810 07:19:40.296616 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.296650 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.296756 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.297013 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.297018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.297397 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.297544 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.297552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.298299 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.298302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.299014 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.299025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.299604 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.299609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.299719 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.300163 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.300183 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.301703 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.303310 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.303321 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.303329 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.303603 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.303839 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.303842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.303844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.303982 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.304429 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.304445 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.304591 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.304762 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.306105 (+ 1343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1091,"thread_start_us":92,"threads_started":1}
I20260810 07:19:40.307492 19569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.190@KRBTEST.COM
I20260810 07:19:40.315091 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.307864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.189:33355 (local address 127.0.0.1:51024)
0810 07:19:40.307971 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.307984 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.308060 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.308287 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.308291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.308459 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.308548 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.308555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.309386 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.309388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.310024 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.310035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.310481 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.310485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.310558 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.310942 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.310955 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.312095 (+ 1140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.313808 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.313811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.313812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.313987 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.314255 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.314256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.314258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.314327 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.314689 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.314691 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.314767 (+ 76us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.314871 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.314988 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 07:19:40.315366 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.307955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.12.117.189:33355)
0810 07:19:40.308071 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.308084 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.308098 (+ 14us) server_negotiation.cc:407] Connection header received
0810 07:19:40.308142 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.308145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.308189 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.308271 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.308680 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.309280 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.310139 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.310371 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.310449 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.312236 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.312239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.312241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.312266 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.313708 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.314080 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.314084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.314086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.314175 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.314410 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.314413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.314414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.314595 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.314693 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.314925 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.315050 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 07:19:40.315811 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.308156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56056 (local address 127.12.117.190:33233)
0810 07:19:40.308250 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.308253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.308282 (+ 29us) server_negotiation.cc:407] Connection header received
0810 07:19:40.308394 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.308397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.308429 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.308481 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.309054 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.309626 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.310486 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.310739 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.310831 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.312994 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.312998 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.313000 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.313027 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.314261 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.314630 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.314633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.314635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.314717 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.314990 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.314993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.314995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.315190 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.315426 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.315571 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.315677 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:19:40.315956 19569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.188@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.117.189@KRBTEST.COM
I20260810 07:19:40.323488 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.316377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57320)
0810 07:19:40.316479 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.316492 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.316563 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.316775 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.316778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.316912 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.316979 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.316984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.317712 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.317715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.318480 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.318490 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.318881 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.318884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.318963 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.319310 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.319323 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.320645 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.322224 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.322227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.322229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.322402 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.322634 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.322637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.322639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.322711 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.323023 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.323026 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.323104 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.323185 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.323342 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:19:40.323552 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.316409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.12.117.188:35285)
0810 07:19:40.316486 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.316490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.316529 (+ 39us) server_negotiation.cc:407] Connection header received
0810 07:19:40.316645 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.316648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.316683 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.316748 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.317086 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.317612 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.318565 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.318792 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.318872 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.320773 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.320776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.320779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.320803 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.322122 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.322483 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.322486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.322488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.322559 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.322784 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.322787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.322789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.322939 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.323019 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.323252 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.323397 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":35}
I20260810 07:19:40.324437 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.316788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.12.117.189:33355)
0810 07:19:40.316883 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.316887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.317025 (+ 138us) server_negotiation.cc:407] Connection header received
0810 07:19:40.317086 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.317089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.317134 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.317199 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.317685 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.318408 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.319059 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.319377 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.319491 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.321683 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.321686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.321689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.321713 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.322931 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.323326 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.323329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.323330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.323397 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.323650 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.323654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.323655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.323838 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.323976 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.324235 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.324337 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 07:19:40.327121 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: Bootstrap starting.
I20260810 07:19:40.327286 12758 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 07:19:40.328387 19303 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 3
I20260810 07:19:40.328339 19249 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "00508c8e2cca4cd68a18b782290b334e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56056
I20260810 07:19:40.329757 19569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:40.330534 19569 log.cc:824] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:40.332149 19434 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } attempt: 3
I20260810 07:19:40.332825 19569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: No bootstrap required, opened a new log
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346379, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.190@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:19:40.336733 19569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.336983 19569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1c64fca2654386a598c6491232de97, State: Initialized, Role: FOLLOWER
Aug 10 07:19:40 dist-test-slave-gjrz krb5kdc[19170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786346380, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.117.189@KRBTEST.COM for kudu/127.12.117.188@KRBTEST.COM
I20260810 07:19:40.337236 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.328717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57324)
0810 07:19:40.328908 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.328920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.328999 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.329195 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.329198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.329347 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.329415 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.329420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.330265 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.330268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.330818 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.330826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.331497 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.331500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.331584 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.331970 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.331984 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.333616 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.335608 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.335611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.335613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.335860 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.336190 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.336194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.336196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.336298 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.336751 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.336754 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.336869 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.336975 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.337092 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260810 07:19:40.337319 19569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.337746 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.328784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.12.117.188:35285)
0810 07:19:40.328876 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.328880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.328980 (+ 100us) server_negotiation.cc:407] Connection header received
0810 07:19:40.329065 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.329068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.329111 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.329171 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.329512 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.330162 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.330917 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.331247 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.331758 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.333792 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.333795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.333798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.333847 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.335476 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.335965 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.335968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.335970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.336052 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.336407 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.336410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.336428 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.336648 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.336743 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.337247 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.337626 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 07:19:40.338629 19569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:40.340891 19303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: Bootstrap starting.
I20260810 07:19:40.341387 19441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.332497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.117.188:35285 (local address 127.0.0.1:57328)
0810 07:19:40.332587 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0810 07:19:40.332600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 07:19:40.332665 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 07:19:40.333000 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 07:19:40.333002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 07:19:40.333164 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0810 07:19:40.333231 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.333236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.334046 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.334050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.334847 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 07:19:40.334857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.335512 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.335516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 07:19:40.335606 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.336078 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 07:19:40.336094 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 07:19:40.337377 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 07:19:40.339475 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.339479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.339480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.339720 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.340035 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 07:19:40.340039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 07:19:40.340041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 07:19:40.340129 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 07:19:40.340959 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 07:19:40.340962 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 07:19:40.341053 (+ 91us) client_negotiation.cc:769] Sending connection context
0810 07:19:40.341134 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 07:19:40.341251 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 07:19:40.340924 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.343482 19303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:40.343477 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 07:19:40.344100 19600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:40.344184 19600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:40.344287 19303 log.cc:824] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:40.346249 19303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: No bootstrap required, opened a new log
I20260810 07:19:40.342015 19583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.332546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57328 (local address 127.12.117.188:35285)
0810 07:19:40.332794 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.332800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.332810 (+ 10us) server_negotiation.cc:407] Connection header received
0810 07:19:40.332834 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.332836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.332865 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.332982 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.333333 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.333953 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.334973 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.335385 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.335469 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.337535 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.337539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.337541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.337565 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.339349 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.339843 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.339846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.339848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.339931 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.340243 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.340246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.340248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.340667 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.341657 (+ 990us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.341730 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.341831 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":96,"threads_started":1}
I20260810 07:19:40.344671 19445 master_runner.cc:186] Error getting master registration for 127.12.117.190:33233: OK, instance_id { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" instance_seqno: 1786346379867715 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:40.348121 19445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:40.349066 19303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.349285 19303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00508c8e2cca4cd68a18b782290b334e, State: Initialized, Role: FOLLOWER
I20260810 07:19:40.349581 19303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.350106 19434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: Bootstrap starting.
I20260810 07:19:40.350751 19303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:40.350847 19249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 07:19:40.350936 19249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:40.352084 19249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.352607 19434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: Neither blocks nor log segments found. Creating new log.
I20260810 07:19:40.352982 19249 leader_election.cc:288] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [CANDIDATE]: Term 1 election: Requested vote from peers 7e0b72181bfa4ced911e2bbd741a49d1 (127.12.117.189:33355), 0b1c64fca2654386a598c6491232de97 (127.12.117.188:35285)
I20260810 07:19:40.355193 19527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00508c8e2cca4cd68a18b782290b334e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0b1c64fca2654386a598c6491232de97"
I20260810 07:19:40.355437 19527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:40.356659 19527 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00508c8e2cca4cd68a18b782290b334e in term 1.
W20260810 07:19:40.356712 19622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:40.356880 19622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:40.357059 19213 leader_election.cc:302] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00508c8e2cca4cd68a18b782290b334e, 0b1c64fca2654386a598c6491232de97; no voters:
I20260810 07:19:40.357456 19631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.357640 19631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.359400 19434 log.cc:824] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: Log is configured to *not* fsync() on all Append() calls
I20260810 07:19:40.355204 19187 master_runner.cc:186] Error getting master registration for 127.12.117.189:33355: OK, instance_id { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" instance_seqno: 1786346380081904 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 07:19:40.360193 19187 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 07:19:40.360365 19631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 07:19:40.360518 19631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [term 1 LEADER]: Becoming Leader. State: Replica: 00508c8e2cca4cd68a18b782290b334e, State: Running, Role: LEADER
I20260810 07:19:40.360886 19631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.361737 19434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: No bootstrap required, opened a new log
I20260810 07:19:40.362941 19631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00508c8e2cca4cd68a18b782290b334e. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.363112 19631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: This master's current role is: LEADER
W20260810 07:19:40.364459 19215 tablet.cc:2369] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 07:19:40.365545 19434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.365744 19434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0b72181bfa4ced911e2bbd741a49d1, State: Initialized, Role: FOLLOWER
I20260810 07:19:40.366043 19434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } }
I20260810 07:19:40.366545 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.366829 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.367223 19434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 07:19:40.367544 19388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "00508c8e2cca4cd68a18b782290b334e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7e0b72181bfa4ced911e2bbd741a49d1"
I20260810 07:19:40.367753 19388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 07:19:40.368952 19388 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00508c8e2cca4cd68a18b782290b334e in term 1.
W20260810 07:19:40.372442 19663 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 07:19:40.372560 19663 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 07:19:40.387120 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.361169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.12.117.189:33355)
0810 07:19:40.361268 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.361271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.373020 (+ 11749us) server_negotiation.cc:407] Connection header received
0810 07:19:40.373180 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.373183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.373225 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.373272 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.378082 (+ 4810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.378815 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.381706 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.382067 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.382139 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.383728 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.383733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.383736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.383761 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.385314 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.385780 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.385784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.385786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.385869 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.386161 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.386164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.386166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.386369 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.386508 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.386844 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.386968 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 07:19:40.393549 19572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.357756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.12.117.188:35285)
0810 07:19:40.357850 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.357854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.370141 (+ 12287us) server_negotiation.cc:407] Connection header received
0810 07:19:40.370915 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.370918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.370977 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.371027 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.374226 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.374900 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.379962 (+ 5062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.380298 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.380369 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.388287 (+ 7918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.388291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.388294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.388319 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.391575 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.392055 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.392059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.392061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.392147 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.392432 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.392435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.392437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.392647 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.392768 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.393294 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.393403 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 07:19:40.395589 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:40.360293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56060 (local address 127.12.117.190:33233)
0810 07:19:40.360400 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.360405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.370353 (+ 9948us) server_negotiation.cc:407] Connection header received
0810 07:19:40.370640 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.370643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.370691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.370741 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.376065 (+ 5324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.376808 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.380504 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:40.380835 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:40.380902 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:40.388288 (+ 7386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 07:19:40.388292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 07:19:40.388294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 07:19:40.388320 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 07:19:40.390650 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.394274 (+ 3624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.394277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.394279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.394369 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 07:19:40.394668 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 07:19:40.394672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 07:19:40.394673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 07:19:40.394878 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 07:19:40.394990 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:40.395332 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 07:19:40.395444 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 07:19:40.397569 19638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 07:19:40.398483 19638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 07:19:40.410862 19527 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [term 1 FOLLOWER]: Refusing update from remote peer 00508c8e2cca4cd68a18b782290b334e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:40.411541 19604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:40.411738 19388 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [term 1 FOLLOWER]: Refusing update from remote peer 00508c8e2cca4cd68a18b782290b334e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 07:19:40.412969 19604 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 07:19:40.416687 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00508c8e2cca4cd68a18b782290b334e. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.416671 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00508c8e2cca4cd68a18b782290b334e. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.416790 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.416790 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.423659 19604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.423790 19604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:40.424737 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.424844 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.425173 19669 mvcc.cc:205] Tried to move back new op lower bound from 7316874774154768384 to 7316874773962997760. Current Snapshot: MvccSnapshot[applied={T|T < 7316874774154768384}]
I20260810 07:19:40.425320 19676 mvcc.cc:205] Tried to move back new op lower bound from 7316874774154768384 to 7316874773962997760. Current Snapshot: MvccSnapshot[applied={T|T < 7316874774154768384}]
I20260810 07:19:40.429378 19638 catalog_manager.cc:1347] Generated new cluster ID: 779a721b433a43c39d8815c9e4edbf31
I20260810 07:19:40.429427 19638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 07:19:40.430704 19631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.430802 19631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e [sys.catalog]: This master's current role is: LEADER
I20260810 07:19:40.431761 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00508c8e2cca4cd68a18b782290b334e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00508c8e2cca4cd68a18b782290b334e" member_type: VOTER last_known_addr { host: "127.12.117.190" port: 33233 } } peers { permanent_uuid: "7e0b72181bfa4ced911e2bbd741a49d1" member_type: VOTER last_known_addr { host: "127.12.117.189" port: 33355 } } peers { permanent_uuid: "0b1c64fca2654386a598c6491232de97" member_type: VOTER last_known_addr { host: "127.12.117.188" port: 35285 } } }
I20260810 07:19:40.431855 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 07:19:40.432296 19673 mvcc.cc:205] Tried to move back new op lower bound from 7316874774154768384 to 7316874773962997760. Current Snapshot: MvccSnapshot[applied={T|T < 7316874774154768384}]
I20260810 07:19:40.458992 19638 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 07:19:40.460176 19638 catalog_manager.cc:1504] Loading token signing keys...
I20260810 07:19:40.476065 19638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00508c8e2cca4cd68a18b782290b334e: Generated new TSK 0
I20260810 07:19:40.476833 19638 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 07:19:40.514330 19572 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.511356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57340 (local address 127.12.117.188:35285)
0810 07:19:40.511482 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.511487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.513183 (+ 1696us) server_negotiation.cc:407] Connection header received
0810 07:19:40.513400 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.513404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.513460 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.513519 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.514197 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57340: BlockingRecv error: recv got EOF from 127.0.0.1:57340 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:19:40.516391 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.512565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51062 (local address 127.12.117.189:33355)
0810 07:19:40.512695 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.512699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.515065 (+ 2366us) server_negotiation.cc:407] Connection header received
0810 07:19:40.515400 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.515403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.515453 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.515503 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.516280 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51062: BlockingRecv error: recv got EOF from 127.0.0.1:51062 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 07:19:40.517146 19310 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.511872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.12.117.190:33233)
0810 07:19:40.511991 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.511995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.514878 (+ 2883us) server_negotiation.cc:407] Connection header received
0810 07:19:40.515659 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.515662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.515714 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.515766 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.517035 (+ 1269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56074: BlockingRecv error: recv got EOF from 127.0.0.1:56074 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 07:19:40.521860 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.520278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.12.117.189:33355)
0810 07:19:40.520374 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.520378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.520906 (+ 528us) server_negotiation.cc:407] Connection header received
0810 07:19:40.521130 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.521134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.521211 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.521265 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.521770 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51064: BlockingRecv error: recv got EOF from 127.0.0.1:51064 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 07:19:40.526505 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.525278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.12.117.189:33355)
0810 07:19:40.525384 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.525389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.525691 (+ 302us) server_negotiation.cc:407] Connection header received
0810 07:19:40.525878 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.525881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.525925 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.525968 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.526426 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51066: BlockingRecv error: recv got EOF from 127.0.0.1:51066 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 07:19:40.531901 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.530478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51082 (local address 127.12.117.189:33355)
0810 07:19:40.530567 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.530571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.531062 (+ 491us) server_negotiation.cc:407] Connection header received
0810 07:19:40.531240 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.531243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.531327 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.531375 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.531824 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51082: BlockingRecv error: recv got EOF from 127.0.0.1:51082 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 07:19:40.539839 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.538462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.12.117.189:33355)
0810 07:19:40.538588 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.538591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.539026 (+ 435us) server_negotiation.cc:407] Connection header received
0810 07:19:40.539192 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.539195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.539237 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.539311 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.539760 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51098: BlockingRecv error: recv got EOF from 127.0.0.1:51098 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 07:19:40.550952 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.549792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.12.117.189:33355)
0810 07:19:40.549890 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.549894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.550135 (+ 241us) server_negotiation.cc:407] Connection header received
0810 07:19:40.550299 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.550302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.550354 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.550447 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.550853 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51112: BlockingRecv error: recv got EOF from 127.0.0.1:51112 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 07:19:40.572227 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.571344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.12.117.189:33355)
0810 07:19:40.571436 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.571440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.571472 (+ 32us) server_negotiation.cc:407] Connection header received
0810 07:19:40.571516 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.571519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.571563 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.571644 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.572128 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51118: BlockingRecv error: recv got EOF from 127.0.0.1:51118 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260810 07:19:40.606873 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.605926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.12.117.189:33355)
0810 07:19:40.606063 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.606068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.606158 (+ 90us) server_negotiation.cc:407] Connection header received
0810 07:19:40.606290 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.606293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.606334 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.606409 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.606795 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51120: BlockingRecv error: recv got EOF from 127.0.0.1:51120 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 07:19:40.673791 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.672761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51134 (local address 127.12.117.189:33355)
0810 07:19:40.672890 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.672895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.673037 (+ 142us) server_negotiation.cc:407] Connection header received
0810 07:19:40.673174 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.673177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.673220 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.673284 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.673705 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51134: BlockingRecv error: recv got EOF from 127.0.0.1:51134 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 07:19:40.804581 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:40.803639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.12.117.189:33355)
0810 07:19:40.803751 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:40.803754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:40.803915 (+ 161us) server_negotiation.cc:407] Connection header received
0810 07:19:40.804027 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:40.804029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:40.804075 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:40.804153 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:40.804489 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51144: BlockingRecv error: recv got EOF from 127.0.0.1:51144 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260810 07:19:41.063623 19443 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 07:19:41.062547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.12.117.189:33355)
0810 07:19:41.062681 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.062686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.062770 (+ 84us) server_negotiation.cc:407] Connection header received
0810 07:19:41.062917 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.062921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.062970 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.063037 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 07:19:41.063516 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51150: BlockingRecv error: recv got EOF from 127.0.0.1:51150 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 07:19:41.345346 19600 catalog_manager.cc:1259] Loaded cluster ID: 779a721b433a43c39d8815c9e4edbf31
I20260810 07:19:41.345418 19600 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: loading cluster ID for follower catalog manager: success
I20260810 07:19:41.347162 19600 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: acquiring CA information for follower catalog manager: success
I20260810 07:19:41.348296 19600 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b1c64fca2654386a598c6491232de97: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:41.373613 19663 catalog_manager.cc:1259] Loaded cluster ID: 779a721b433a43c39d8815c9e4edbf31
I20260810 07:19:41.373672 19663 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: loading cluster ID for follower catalog manager: success
I20260810 07:19:41.375154 19663 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: acquiring CA information for follower catalog manager: success
I20260810 07:19:41.376272 19663 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e0b72181bfa4ced911e2bbd741a49d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 07:19:41.581543 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:41.577625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.12.117.189:33355)
0810 07:19:41.577874 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.577879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.577928 (+ 49us) server_negotiation.cc:407] Connection header received
0810 07:19:41.578105 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.578108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.578237 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.578333 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:41.578597 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.579119 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.580091 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.580324 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.580412 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:41.580726 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:41.581168 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:41.581278 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:41.581342 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 07:19:41.581398 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260810 07:19:41.587606 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:41.584348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.12.117.188:35285)
0810 07:19:41.584560 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.584564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.584597 (+ 33us) server_negotiation.cc:407] Connection header received
0810 07:19:41.584671 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.584673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.584776 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.584859 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:41.585120 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.585617 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.586406 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.586618 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.586680 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:41.586911 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:41.587252 (+ 341us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:41.587371 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:41.587404 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 07:19:41.587448 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260810 07:19:41.594063 19814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:41.590281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57350 (local address 127.12.117.188:35285)
0810 07:19:41.590372 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.590376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.590830 (+ 454us) server_negotiation.cc:407] Connection header received
0810 07:19:41.590920 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.590923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.590967 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.591025 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:41.591410 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.592141 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.593042 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.593325 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.593428 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:41.593688 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:41.593753 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:41.593811 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:41.593907 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 07:19:41.593954 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 07:19:41.594414 19800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:41.590196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.12.117.189:33355)
0810 07:19:41.590397 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.590401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.590764 (+ 363us) server_negotiation.cc:407] Connection header received
0810 07:19:41.590913 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.590916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.590962 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.591025 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:41.591314 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.592082 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.593093 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.593381 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.593489 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:41.593713 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:41.593790 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:41.593856 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:41.594216 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 07:19:41.594272 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260810 07:19:41.594911 19828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 07:19:41.590430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.12.117.190:33233)
0810 07:19:41.590672 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 07:19:41.590675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 07:19:41.590687 (+ 12us) server_negotiation.cc:407] Connection header received
0810 07:19:41.590761 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 07:19:41.590764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 07:19:41.590897 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 07:19:41.591011 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 07:19:41.591202 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.591802 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.592806 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 07:19:41.593130 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 07:19:41.593217 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 07:19:41.593743 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 07:19:41.594032 (+ 289us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 07:19:41.594624 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0810 07:19:41.594658 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 07:19:41.594714 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":67,"threads_started":1}
I20260810 07:19:41.597465 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19187
I20260810 07:19:41.618402 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19316
I20260810 07:19:41.629338 12758 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task3eVEGZ/build/debug/bin/kudu with pid 19445
2026-08-10T07:19:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3207 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6424 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (283817 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST